No weights changed, one layer block run twice: Qwen3-1.7B climbs from 42.30% to 79.61% on GSM8K

The student still runs each layer once in the original order, yet its accuracy rises, so the supervision can come from extra computation the same model performs rather than from a larger external teacher. Earlier self-distillation obtained more useful teacher predictions by giving the teacher additional context or instructions, while the teacher itself stayed on the standard path. On GSM8K at a 400-token thinking budget, Qwen3-1.7B goes from 42.30% to 79.61%, and Phi-4-reasoning-plus from 89.54% to 94.92%.

The procedure first searches labeled questions for which contiguous block of layers to repeat, freezes the looped model as the teacher, and trains the student to match the teacher's next-token distribution on prefixes the student itself generated. Search and distillation alternate, with each round rebuilding the teacher from the updated student until validation no longer identifies a useful looped route or the round limit is reached. On GSM8K, 100 training questions are held out for validation and the remaining 7,373 are used for distillation, with route search using labels from 1,000 of those questions; each round runs 200 updates for at most ten rounds.

Under the same thinking budget, the student answers more accurately while generating fewer tokens. Raising accuracy at a fixed budget has typically come with longer reasoning chains. For Qwen3-4B on GSM8K at a 400-token thinking budget, accuracy rises from 74.98% to 89.76% while average total generation falls from 385.04 to 299.98 tokens.

The report is for researchers and engineers working on reasoning-model training and distillation. The demonstrated scope is accuracy on mathematical and logical reasoning tasks, generation length under fixed thinking budgets, label efficiency, ablations, and layerwise answer readout; students trained on GSM8K are evaluated on GSM-Plus and SVAMP without target-task training. Teacher selection needs labeled search questions and a validation set, while distillation itself uses only questions and student-generated responses.

Adopting a looped route requires a strict validation improvement over the standard route, otherwise standard execution is kept; under concise teacher prompting, validation retained the standard route on GSM8K. In the label-efficiency comparison the label counts and update counts are matched, but search compute and access to unlabeled questions differ. The layerwise readout analysis uses fixed ProofWriter contexts and the original readout head, without training layer-specific probes.