Reasoning correction without retraining: injecting the instruction as a new user turn lifts average accuracy from 37.1% to 55.3% across four reasoners
Lead
Across four reasoners and six math benchmarks, injecting a correction instruction as a new user turn at a learned cut position raises average accuracy from 37.1% to 55.3% while the reasoner itself stays frozen.
Story
What makes correction work is not the wording of the instruction but the conversational structure it arrives in: the same sentence placed inside the model's own continuation barely changes its course, while delivering it as a new user turn does. Earlier self-correction methods either fine-tune the reasoner to produce a revised answer or train an external critic or verifier to give step-level feedback, and typically optimize an LLM-scale component. Under a fixed generation budget, the tested inline injections (reflection tokens, explicit instructions, even an unrelated word) failed to reliably redirect the rollout, whereas the same text delivered across a turn boundary yielded substantially higher continuation success rates.
The system trains only two lightweight heads: one decides where to cut, the other decides which rollouts need correcting, and the reasoner stays frozen throughout. Earlier correction methods typically train an LLM-scale component; here a 130K-parameter MLP head on a frozen encoder learns cut positions using only the final-answer correctness of regenerated rollouts. The position policy trains for 30 steps on DAPO-17K with cuts restricted to paragraph boundaries, and the 66K-parameter gate trains once on 2,874 correctness-labeled rollouts and is reused across all reasoners.
The correction gains hold consistently across reasoners and benchmarks, and carry over to a reasoner and a subject it was never trained on. Earlier methods are typically tied to a specific reasoner and require training a correction component per model. Across four reasoners from 4B to 32B and six math benchmarks, UPTURN@16 beats Base@32 in average accuracy on every benchmark; transferred position policies improve Avg over Base by 17.9 points versus 19.3 for target-trained ones, and transfer to GPQA-Diamond without retraining.
What to watch
The correction pipeline is validated in a fixed-budget prefix-regeneration setting: a selected prefix is retained, a fixed instruction is injected as a new user turn, and the frozen reasoner regenerates the suffix. Evaluation covers four reasoners and six math benchmarks, plus cross-reasoner transfer and transfer to GPQA-Diamond without retraining. It is aimed at researchers and engineers working on inference-time correction and test-time compute allocation.
The study focuses on reasoning tasks whose outcomes can be checked automatically, and on a single, deliberately minimal form of intervention. Richer interventions, and settings where success is harder to verify, may call for different choices of what to inject and where, which the authors leave to future work. The advantage narrows as the generation budget grows, and on the two Qwen3 reasoners Base@32 matches or slightly exceeds it at the largest budget.