Small-model reasoning distillation drops the fixed question set: 1,000 trajectories lift a 4B model from 62.9 to 75.3 on IMO-AnswerBench

The training data for long-chain reasoning distillation can be re-selected as the student changes, instead of being fixed before training starts. Existing distillation pipelines generate and filter a fixed collection of teacher trajectories and then train on that same data repeatedly; other methods prioritize questions the current student fails, but react to errors without explicitly modeling how capability changes. On Qwen3.5-4B, with the same 1,000 training examples, the same Stage 1 checkpoint and the same training budget, BoundaryEvo beats random selection and LLM2LLM on all eleven reasoning benchmarks, exceeding them by 8.7 and 6.4 points on the mathematical average and by 2.5 and 1.2 points on general reasoning.

The procedure has two steps: first give the student basic reasoning ability, then let the questions follow its capability boundary. Stage 1, capability priming, directly distills 100 teacher-generated long reasoning trajectories with no extra model-based judging or capability-gap filtering; Stage 2 has the student and a fixed expert independently solve the same candidate questions each round, and an analyzer judges capability gap, data quality and question validity together, keeping only questions the expert solves reliably and the student has not mastered. Stage 2 selects from 1,999 candidate questions and trains on 1,000 examples over three turns (100 plus 300 times 3), updating the student by LoRA supervised fine-tuning each turn, replaying unresolved questions up to a quota and filling the rest of the budget with quality-ranked new questions.

The reported setting is tool-free long-trajectory distillation: a compact student, a fixed candidate pool and expert, and boundary re-estimation after each student update. It is aimed at researchers and engineers working on small-model reasoning distillation and training-data selection. The authors state that code and data will be released, along with question-ID manifests, exclusion counts and the full evaluation configuration.

The authors report that the 2B student saturates earlier and gains less stably across turns, suggesting limited model capacity can constrain long-term evolution. The data-efficiency comparison measures example counts only and excludes student and expert inference, analyzer judging and teacher-rollout generation, and dataset definitions and filtering policies differ across papers. The trajectory-length analysis is observational and does not establish causality.