Across six pretrained language models, residual streams favor their own endpoint early while competitor sets shrink with changing membership
By comparing each context's intermediate residual states with its own final state and an empirical bank of final states from other contexts, the study finds across Gemma-2B, Gemma-7B, Qwen2.5-1.5B, Qwen2.5-7B, Mistral-7B, and Llama-3-8B that the own endpoint becomes preferable to the average alternative at the earliest measured layer while many individual endpoints remain closer, that competitor sets generally shrink with depth yet show both entries and exits, and that directional alignment can improve while Euclidean distance to the final state changes little; the authors add a high-dimensional model separating norm, alignment, and endpoint geometry and prove that a straight path toward the own endpoint cannot introduce new competitors.