Same attention heads, new task, new role: one internal activation edit changes which fact the model reads and how it influences later layers

The same attention heads take on different roles across tasks, and that difference shows up in how they influence later computation, not only in whether they are reused. Earlier work could identify attention heads reused across tasks, but identifying a shared head does not explain what it does differently in each task. In Falcon-H1-1.5B, with graph facts held fixed and only the instruction changed from Compose path to Read direct edge, H1's mean influence on H2 moves from −0.0145 to 0.00101, its influence on H3 rises from 0.0107 to 0.0238, and its influence on the later residual site falls from 0.0446 to 0.0346.

Task context changes both which graph facts the same heads respond to and how their outputs affect later components. With the written instruction still Read fact 1, H2 and H3 were most responsive to Fact 1; after the early activation edit they become most responsive to Fact 2. In Gemma-2-9B, Fact 2's response share rises by 78.9 percentage points at H2 and 97.1 at H3; Qwen3-14B's corresponding increases are 68.7 and 91.8 percentage points, and all four corrected intervals exclude zero.

A linear response map selected for the current internal state predicts the downstream effect of a separate factual intervention better than averaging the maps from before and after the edit. The prior comparison averaged the unedited and task-edited response maps and used that single averaged map to predict downstream responses. In Gemma-2-9B the task-specific map lowers mean squared prediction error by 53.6% relative to the averaged map, and in Qwen3-14B by 5.0%, with both corrected difference intervals excluding zero.

The results cover five open-weight models spanning four families and 1.5–14B parameters; the influence-versus-importance comparison is done in Gemma, Qwen, Llama and Falcon-H1-7B, while factual interventions, internal task edits and response prediction use Gemma and Qwen without additional training, and the conditional-response analysis uses Gemma and Qwen fine-tuned on graph tasks. Each influence value describes a response transmitted through the intervening network, including components outside the selected sites, and its interpretation is tied to the chosen activation direction and perturbation size. The prediction analysis uses the measured source displacement and a response map estimated for the same prompt, so the improvement shows the benefit of matching the internal task state within that setting. The authors propose as a next step testing whether influence measurements predict responses on new graphs and instructions beyond the prompts on which they were estimated, and whether a finer description of components explains which interactions transfer.

Each influence value describes a response over a finite perturbation range rather than an infinitesimal derivative or an isolated direct connection. The response-map improvement is obtained within the same prompt, using the measured source displacement and a map estimated for that prompt. The conditional-response finding applies to Gemma and Qwen fine-tuned on graph tasks and evaluated under the AND instruction. In the exploratory comparison restricted to the three heads, influence angles are smaller than importance angles in Gemma and Falcon-H1-7B and larger in Qwen and Llama, but these are descriptive point estimates and all four angle-difference intervals include zero. The authors also note that whether influence measurements predict responses beyond the prompts on which they were estimated remains an open question.