Reasoning post-training without the full corpus: two-stage contraction cuts the optimization budget to 0.22%
Synopsis
Reasoning post-training no longer feeds every teacher chain-of-thought to the student at equal weight. Scaling the corpus had raised training cost without proportional gains. Here Monte Carlo reasoning trees estimate process uncertainty at intermediate states to contract the corpus into a teacher-informed pool, and online routing during supervised fine-tuning concentrates updates on what the student has not yet learned, cutting the optimization budget to 0.22–1.91% of full-data SFT across reasoning benchmarks and model families while matching or improving performance.
Interpretation
The amount of reasoning supervision can be compressed sharply while student reasoning scores hold or rise. The prior path scaled teacher-generated chain-of-thought corpora, raising training cost without proportional performance gains. On DAPO-17K with Qwen3-8B as teacher and Qwen3.5-4B and 9B as students, the method uses 0.22–1.91% of the full-SFT optimization budget and matches or exceeds it on most benchmarks.
Which trajectories to keep is decided by how much intermediate reasoning states diverge, not by final-answer correctness. Earlier data selection relied on correctness, difficulty, or quality, treating verified-correct chains as equally valuable. Among the same verified-correct trajectories and at matched pool sizes, uncertainty-based selection beats random selection by 1.6–4.5% average performance.
Teacher-side uncertainty profiling and learner-side routing each contribute, and their combination performs best. Either stage alone improves accuracy, but the two act on different parts of the pipeline. Ablations show teacher-side uncertainty improves the informativeness of candidate supervision while learner-aware routing concentrates updates on trajectories the student has not yet captured.
Perspective
The method targets reasoning post-training by chain-of-thought distillation; teacher-side profiling needs only prefix-conditioned sampling and outcome verification, so it applies to open-weight teachers and to API-only closed-source teachers. Demonstrated students include Qwen3.5-4B and 9B, Gemma-4-E4B-IT, Llama-3.2-3B-Instruct, and Mistral-7B-Instruct; in the weak-to-strong setting the teacher is Qwen3-1.7B and the students are Qwen3.5-4B and Qwen3-14B.
Efficiency figures vary with the contraction ratio, and the lowest reported optimization budget is 0.22%, about 450 times less training exposure than full-data SFT; performance stays stable under more aggressive contraction but fluctuates across benchmarks. Once built, the teacher-side profile can be shared across students, and its value depends on whether prefix sampling and outcome verification are available.