Repeated Assertions Are Not Accumulating Evidence: LLM Inference Dynamics Converge to Query-Dependent Limits
Related research and updates
Synopsis
The authors introduce a theoretical framework for contextual influence through inference dynamics, showing that under a controlled single-round setting repeated contextual assertions do not cause unbounded drift but instead drive internal inference trajectories to stable, query-dependent limits, so repetition does not act as accumulating evidence and may fail to change a prediction or make a flip inevitable; they validate representation-level dynamics against output-level answer shifts across six models and several QA benchmarks.
Figure 1 : Heterogeneous effects of contextual repetition on model predictions. Illustrative examples showing qualitatively distinct behaviors of LLMs under repeated contextual assertions, including immediate answer flips, delayed changes, early saturation, and complete invariance to repetition.
arXivInterpretation
The paper recasts contextual influence as a problem of internal inference dynamics and provides a formal framework for how repeated contextual signals shape representation-level inference trajectories, moving beyond answer-level prediction changes. Prior work predominantly assessed contextual sensitivity through finite prompt manipulations and observable answer flips; this work turns to representation-level trajectories and explicitly models repetition as a growing contextual signal. The paper provides a problem formulation and theoretical framework, defining the target position and loop template in a controlled single-round setting as the basis for the subsequent convergence analysis and experiments.
The paper proves that under repeated contextual assertions internal inference trajectories converge to stable, query-dependent limits rather than exhibiting unbounded drift, showing that repetition does not function as accumulating evidence during inference. This challenges the common intuition that repeatedly presenting a candidate answer progressively steers the prediction toward that answer and eventually overwhelms the original query signal. Under Assumption 3.1 (causality, RoPE as the only positional mechanism, finite operator norms, uniformly bounded residual streams, Lipschitz normalization and FFN on bounded sets, infinite context window), convergence guarantees follow from a finite trigonometric-polynomial expansion of RoPE logits, Cesàro limits, and a layer-wise induction.
The paper provides representation-to-prediction alignment evidence indicating when a prediction change becomes inevitable and when it is provably unattainable under unbounded repetition. It directly connects representation-level convergence to observable answer-preference changes and provides an estimator that approximates the limit from a single forward pass at a large repetition length. Across OpenBookQA, MINTAKA, SimpleQA and other benchmarks with six models, the representation-level estimator and the forward-computed answer shift are reported to align closely along the diagonal, and the KL divergence between next-token distributions plateaus as repetition grows.
On sycophancy and persuasion benchmarks including SYCON, Farm, BeHonest and SycoEval, the representation-based stability measure shows trends broadly consistent with prior amplification metrics. It offers a mechanistic account of why these effects arise and when they saturate, are delayed, or fail to materialize, complementing existing answer-level evaluations. Table 3 reports model-level average predicted answer-shift magnitude alongside the amplification metric, with trends broadly consistent with prior evaluations except for a small number of gray-highlighted cases.
Perspective
The results apply to a controlled single-round setting: a fixed query followed by repeated instances of an identical contextual assertion, without additional evidence, reasoning steps or multi-turn interaction, and with models satisfying Assumption 3.1 (causality, RoPE as the only positional mechanism, finite operator norms, uniformly bounded residual streams, Lipschitz normalization and FFN on bounded sets, infinite context window). Within this setting, the framework can indicate whether a given query-model pair is stable, readily flipped, or saturating under repetition, and can estimate the limiting answer shift from a single forward pass at a large repetition length, offering guidance for prompt design, evaluation robustness and safety analysis.
Readers should still watch: conclusions rest on a controlled single-round setting with a single repeated assertion and Assumption 3.1, so extension to multi-turn dialogue, diverse prompt structures and other positional mechanisms remains to be tested; the estimator uses a single-layer first-order treatment with the actual large-repetition residual stream as a surrogate for the limit, and its approximation error across depths and models deserves further characterization; a small number of gray-highlighted cases in Table 3 do not fully match prior amplification trends, and their causes remain open; and some numerical details presented as tables and figures in the main text need to be read together with the appendix and figures.