Reasoning models learn where to stop: LoCo cuts math completions by 41.7%–45.4% by training only the continue-or-conclude choice
Synopsis
Large reasoning models often keep deliberating after enough information is available, raising decoding latency and memory cost. LoCo moves the compression decision to individual reasoning states: it finds candidate overthinking states in the model's own traces, freezes that prefix, regenerates shorter continuations whose answers verify as correct, and learns a continue-or-conclude preference with DPO. Across five math benchmarks, completion length drops 41.7%–45.4% while accuracy holds or improves; trained on math only, the same model cuts length by 23.4%–36.3% on three code benchmarks.
Interpretation
The supervision for reasoning compression moves from the whole reasoning trajectory to a single reasoning state, so the model learns whether to keep reasoning or conclude at that state. Earlier trajectory-level methods compare the length of complete solutions, and fine-grained methods delete or shorten redundant content within a trajectory; both treat the trajectory itself as the object of compression and give less direct guidance on whether the current state still needs more reasoning. On AMC23, AIME24 and AIME25, LoCo reduces completion length by 41.7%, 45.4% and 45.2%, while a global-preference variant with the same query set and pairing thresholds reduces it by only 5.2%, 11.1% and 15.8%.
The procedure first screens suspicious states with the model's own confidence, then has a stronger judge model confirm whether the next step merely restates what is already established. Low confidence can come from a genuinely hard sub-problem, and high volatility can accompany a legitimate change in reasoning, so confidence alone cannot separate redundant deliberation from real difficulty. Screening uses only information available by the end of that step and truncation begins only from the next step, so the detector never sees the continuation it removes; flagged states are re-checked by Qwen3-30B, and only states whose next step is judged redundant are kept.
At each retained state the prefix is frozen and continuations are resampled; the shortest one whose answer is symbolically equivalent to the gold answer is paired against the original continuation for DPO training under the same prefix. Both branches of a preference pair share an identical reasoning prefix, so the comparison falls only on how to write forward from that state rather than on the overall length of two complete solutions. Training data come entirely from traces DeepSeek-R1-Distill-Qwen-7B generates on math-only questions, with a cap on pairs per question and a minimum token saving required of the chosen continuation; evaluation covers five math benchmarks plus MBPP, LiveCodeBench and CodeContests.
Perspective
The demonstrated setting is DeepSeek-R1-Distill-Qwen-7B trained with a LoRA adapter, evaluated on five math benchmarks and on MBPP, LiveCodeBench and CodeContests, with eight completions sampled per problem.
The cross-domain gain is interpreted as reshaping the model's preference over reasoning length rather than injecting task-specific knowledge, and that interpretation is an inference; the weaker external-continuation variant is likewise attributed to a possible distribution-alignment effect.