# Attention-Guided Layer Selection

Contrastive decoding's DoLa picks its premature layer from differences in output vocabulary distributions; Sakai and colleagues instead use self-attention distributions as the selection signal, introducing Attention-JSD, Attention-Entropy-Max and Attention-Entropy-Min while leaving the DoLa framework intact. On TruthfulQA, LLaMA-7B with Attn-Ent-Min reaches MC2 62.8% and MC3 36.5%, above DoLa's 56.5% and 30.5%; on LLaMA-33B and 65B, however, DoLa keeps the highest MC1, so the gains concentrate on multi-answer metrics.

Contrastive decoding's DoLa picks its premature layer from differences in output vocabulary distributions; Sakai and colleagues instead use self-attention distributions as the selection signal, introducing Attention-JSD, Attention-Entropy-Max and Attention-Entropy-Min while leaving the DoLa framework intact. On TruthfulQA, LLaMA-7B with Attn-Ent-Min reaches MC2 62.8% and MC3 36.5%, above DoLa's 56.5% and 30.5%; on LLaMA-33B and 65B, however, DoLa keeps the highest MC1, so the gains concentrate on multi-answer metrics.