DoLa swaps logits for attention when picking layers: LLaMA-7B MC2 reaches 62.8%, while larger models keep MC1 with DoLa

Which layer contrastive decoding contrasts against can now be decided from the model's internal attention distributions rather than only from probability differences over the output vocabulary. DoLa selects, at each token, the premature layer whose output distribution has the largest Jensen–Shannon divergence from the final layer, a criterion that does not directly use the structural information carried by attention. On TruthfulQA, LLaMA-7B with Attn-Ent-Min reaches MC2 62.8% and MC3 36.5% against DoLa's 56.5% and 30.5%; on LLaMA-13B, Attn-JSD is best on all three of MC1, MC2 and MC3.

All three strategies first average the attention weights of every head at the current position over past tokens, producing one normalized attention distribution that then ranks the candidate premature layers. Attention-JSD takes the layer whose attention distribution diverges most from the final layer, Attention-Entropy-Max takes the layer with the highest attention entropy, and Attention-Entropy-Min takes the lowest, where entropy measures how broadly attention is spread. Experiments cover six models, LLaMA-7B, 13B, 33B and 65B plus Gemma-7B and Mistral-7B, with LLaMA-33B and 65B run in int8 quantization; candidate layers follow DoLa's buckets, and the bucket is chosen by validation.

Attention-based layer selection and DoLa do not get the same questions right, and on LLaMA-33B the two show complementary score shapes. DoLa has the highest MC1 while the attention methods are slightly higher on MC2 and MC3, and at the sample level DoLa and Attn-Ent-Max are both correct on only 150 questions. On LLaMA-33B, DoLa's top margin is 14.8 and the standard deviation of correct-option scores is 31.3, versus 8.5 and 17.5 for Attn-Ent-Max, corresponding to sharper and flatter distributions.

Evaluation centers on the multiple-choice settings of TruthfulQA and FACTOR, with FACTOR restricted to its Wiki and News subsets; the six main models are LLaMA-7B, 13B, 33B and 65B plus Gemma-7B and Mistral-7B, and the two largest run in int8 quantization. The authors list validation under free-form generation and automatic selection of the candidate-layer bucket as future work.

The candidate premature-layer bucket still has to be chosen in advance with a validation set and is not fully automatic; why certain attention heads are better for factual recall is not yet explained; and on models such as Gemma-2-2B, contrastive decoding falls below the baseline.