Speculative decoding stops picking layers by position: entropy and CKA choose the set, lifting accepted tokens by about 0.085 on Qwen3

Multi-layer hidden-state selection moves from a fixed positional rule to a joint score over information content and geometric redundancy, and the draft model is conditioned on the concatenation of the chosen layers. Prior speculative decoding picked layers with fixed positional heuristics, such as EAGLE2 taking only the last layer and EAGLE3 taking layer 2, the middle layer and layer N-3, without explicitly balancing informativeness against redundancy. On Qwen3-4B-Instruct-2507 the selected three-layer set reaches 4.29 mean accepted tokens on SpecBench, against 4.20 for the EAGLE3 default triple and 4.00 for EAGLE2, across draft-tree sizes of 30 and 60 nodes and temperatures 0.0 and 1.0.

Selection rests on two computable signals: matrix entropy of the concatenated representation measures how much a layer set carries jointly, and CKA transitions between depths mark where the representation changes compartment. Per-layer entropy alone says a layer is informative but not which layers complement each other, so the entropy is computed on the concatenation of a candidate set and CKA transitions steer the expansion. On Qwen3-4B-Instruct-2507 and Llama-3.1-8B-Instruct, per-layer Shannon entropy follows a non-uniform activation wave across depth, entropy gains from pairing layers are uneven, and CKA transitions divide depth into distinct representational compartments.

Calibration first filters by token predictive entropy, keeping hidden states at high-entropy token positions before layer entropy and CKA statistics are computed. Per-layer entropy varies strongly across prompts, especially in the first and last layers, which would make the layer ordering prompt-dependent even though the selected set stays fixed through training and inference. With a retained fraction of 0.30, cross-prompt entropy variance drops by 48.4% in early layers 0-6, 45.8% in middle layers 7-28 and 35.6% in late layers 29-35, while the depth-wise activation-wave shape is preserved.

The report covers layer-selection analysis and EAGLE3-style speculative decoding evaluation on two open models, Qwen3-4B-Instruct-2507 and Llama-3.1-8B-Instruct, evaluated on SpecBench, with draft models trained on distillation data generated from ShareGPT and Ultrachat on 8 H100 GPUs. It suits readers working on inference acceleration and draft-model conditioning who want to judge whether layer-set selection should replace an existing fixed rule. The source lists experimenting with ELSI-generated layer sets on edge devices as future work.

On Llama-3.1-8B-Instruct, adding selected layers brings no further gain, with the average accepted tokens varying by only 0.017 across m=1, 2 and 3, and ELSI (m=3) trailing EAGLE3 by 1.0% on average; the source attributes this smaller headroom to greater interchangeability of that model's decoder layers, visible as broad high-CKA regions. The source also notes that a smaller set of target states is expected to give higher throughput, though this may not be visible on GPUs that are not memory-bound.