Where to inject activation steering: TopoSteer ranks layers by the connectivity length of displacement clouds, skipping held-out trials
Synopsis
Activation steering adds a vector to a large language model's internal representations at inference time, but where to add it has mostly been settled by trial: existing selectors score how separable the two activation classes are or how well individual displacements align with their mean, without ever steering the model to check. TopoSteer instead scores each layer by the total length of the Euclidean minimum spanning tree of its per-example displacement cloud, measuring whether the mean-difference vector can stand in for those displacements. Across five instruction-tuned models and six binary persona-steering tasks it beats LayerNavigator in most model-budget comparisons, but the margin depends on model and layer budget and is smaller and more mixed on Qwen-7B and Qwen-32B.
Interpretation
Layer selection gains a scoring basis that needs no held-out steering runs: instead of asking whether the two activation classes separate, it asks how spread out the cloud of per-example displacements that mean-difference steering averages is. The prior validation-free selector, LayerNavigator, ranks layers with a Fisher-type discriminability term plus a directional-consistency term; the first looks only at class means and within-class scatter, the second is an average of cosines that is unchanged when each displacement is rescaled independently, so neither sees the overall extent of the displacement cloud. Across five instruction-tuned models and six binary persona-steering tasks, under simultaneous injection TopoSteer is higher than LayerNavigator in most model-budget comparisons, ties once and is lower four times; the largest gain is on Qwen-3B, where at a layer budget of 3 alignment probability rises from 61.55 to 66.20.
The scored object is the per-pair displacement vector at each layer in the standardized representation, and the score is the total persistence of that point cloud, equal to the summed edge weights of its Euclidean minimum spanning tree. A minimum spanning tree connects the cloud with the smallest possible total edge weight, so the summed length reads off how spread the cloud is; the score is invariant to a common translation and depends only on the relative arrangement of the residual displacements. The score reuses the same cached activations as the steering vectors, requiring no extra model forward passes, auxiliary training or held-out steering evaluations; the added cost is an offline geometric computation over the extraction data.
Discriminability and connectivity can rank layers in opposite orders, which explains why the two selectors pick different layers. On a one-parameter family that fixes class means and scales within-class residuals, total persistence strictly increases with that scale while LayerNavigator's discriminability term strictly decreases. The result concerns a fixed layer and the discriminability component alone; real layers may also differ along directions where the two scores agree, so the selectors sometimes coincide.
Perspective
The demonstrated scope is six binary persona-steering tasks on five instruction-tuned models (Llama-3-3B, Llama-3-8B, Qwen2.5-3B, Qwen2.5-7B, Qwen2.5-32B), evaluated by average aligned-answer probability and generation perplexity, under both simultaneous injection and greedy sequential injection that re-selects the next layer each round.
The method assumes matched positive and negative examples and currently considers only mean-difference steering, leaving other extractors to future work; multi-layer steering can raise perplexity on some models; the reversal between discriminability and connectivity is proved for a fixed layer, and its across-layer implications remain empirical; a score with a high bridge share may reflect a mode split rather than the arrangement within the cloud.