No weight updates at inference: steering neurons per query lifts seven models from 3B to 32B
Synopsis
Instead of a fixed intervention chosen in advance, Q-NeuS reads each query's own internal activations to decide which feed-forward neurons to amplify, improving every one of seven models from 3B to 32B parameters across three benchmarks.
Interpretation
A large language model can decide at inference time which internal neurons to intervene on for the query in front of it, rather than following an intervention fixed in advance. The two existing routes either shape the input through context engineering or modify model computation through fixed steering, with the steering target or direction identified beforehand, which struggles to fit queries with different knowledge and reasoning needs. Across MMLU-Pro, SuperGPQA and WildSci, on seven models from 3B to 32B parameters, Q-NeuS scores above the unintervened baseline on every model and benchmark, for example lifting Qwen2.5-3B on MMLU-Pro from 31.43 to 38.37.
The mechanism is a lightweight gate that reads the activations a query induces inside the model and outputs a modulation coefficient for each feed-forward neuron. One gate is shared across all Transformer layers, with a learnable layer embedding preserving layer-specific information, and the base model stays frozen while only the gate is optimized. Training uses Neu-GRPO, which optimizes the gate from the relative performance of multiple neuron intervention configurations for the same query; candidate neurons are selected per subject and layer from training data, and inference needs no task or subject labels.
Under cross-benchmark transfer, a policy trained only on MMLU-Pro still improves or matches the baseline on most knowledge-intensive reasoning benchmarks. Against GRPO trained on the same MMLU-Pro data, Q-NeuS does better on most target benchmarks, because GRPO learns one fixed parameter update while Q-NeuS re-decides the intervention from each new query's internal activations. Direct evaluation on GPQA-Main, GPQA-Diamond, SuperGPQA, LiveBench, AGIEval, GSM8K and ARC-Challenge shows consistent transfer on GPQA and SuperGPQA, while LiveBench gains are limited and occasionally negative.
Perspective
The report covers feed-forward neuron modulation at inference: base model parameters stay frozen during training and inference, no task or subject labels are needed at inference, and the gate accounts for roughly 0.32% to 1.46% of backbone parameters. Evaluation spans seven models from 3B to 32B parameters on MMLU-Pro, SuperGPQA and WildSci in-domain, plus direct evaluation on GPQA-Main, GPQA-Diamond, SuperGPQA, LiveBench, AGIEval, GSM8K and ARC-Challenge after training only on MMLU-Pro. It suits researchers and engineers working on inference-time intervention and capability elicitation.
Cross-benchmark transfer is uneven: gains on LiveBench are limited and occasionally negative, which the authors attribute to its larger distribution shift from MMLU-Pro, including coding and data-analysis tasks less represented in subject-oriented training data. Ablations show accuracy dropping clearly when the neuron sampling ratio reaches 90%, which the authors read as excessive sampling reducing the diversity of intervention configurations across rollouts. The case study covers five queries, and the query-specific neuron overlap patterns also vary across Transformer layers.