A small model plays skeptic to a large one: DoubtLess trims Qwen3-32B's chain of thought by 31% without retraining
Synopsis
Reasoning models often keep re-checking after they have reached an answer, stretching the chain of thought. DoubtLess has a small model that shares the large model's tokenizer read the same chain twice, once told to go straight to the answer and once told to double-check every step, and adds the scaled difference of the two logit vectors to the large model's logits at every step, acting only at decoding time with no weight changes. On seven competition-mathematics, graduate-science and programming benchmarks, Qwen3-32B steered by Qwen3-0.6B uses 31% fewer thinking tokens at unchanged accuracy, a larger cut than five inference-time baselines.
Interpretation
A sizeable part of a reasoning model's chain of thought re-derives and re-verifies a result after the answer has appeared, and DoubtLess removes that redundant checking without changing the model's weights. Earlier ways to shorten the chain either retrain each model, ask for brevity in the prompt, ban words such as wait, end the chain by a rule, or add a fixed vector to hidden states; these can cut useful reasoning along with the redundant checks. Across seven benchmarks of competition mathematics, graduate-level science and competitive programming, Qwen3-32B steered by Qwen3-0.6B uses 31% fewer thinking tokens at unchanged accuracy, more than five inference-time baselines; on AIME 2025, banning the words that open another round of checking costs 14 points while DoubtLess costs none.
The steering direction comes from the difference between two stances of one small model on the same context, recomputed from the current text at every step. The small model reads the chain written so far once as a brief thinker told to go straight to the answer and once as a verbose thinker told to double-check every step, and the difference of the two logit vectors, scaled by a strength, is added to the large model's logits. The contrast acts only inside the chain of thought, the answer is decoded by the large model alone, the logits for the end-of-thinking tag and the end-of-sequence token keep the large model's own values, and candidates stay within the large model's own top-p set; a random-target control that assigns the same adjustments to randomly chosen tokens changes thinking length by 1% across the seven benchmarks.
The removed tokens lie mainly after the correct answer first appears, not in the derivation itself. Splitting each chain at the first appearance of the answer, DoubtLess halves the thinking after that point and cuts the thinking before it by a fifth. On correct AIME 2025 chains, the two next-token distributions of the small model diverge seven times more after the answer than before it; on AIME 2025 problem 11 an algebraic check restores a wrongly excluded root, showing that necessary checks remain.
Perspective
The demonstrated setting is Qwen3-32B steered by Qwen3-0.6B on seven English benchmarks in single-turn decoding, plus a self-debugging coding agent on HumanEval+ and the EXAONE-4.0 and Granite-4.2 families; one strength value is used for every model pair and benchmark.
Accuracy falls once the strength grows past a point, dropping 7.8 points at a larger strength on AIME 2025; larger small models remove close to 60% of thinking tokens but likewise pass the point where accuracy holds. On the coding agent, DEER removes 41% of thinking and raises the pass rate by 4.5 points, more than DoubtLess's 20% and 2.0 points.