# Protocol Distillation

A single large model that both generates and evaluates its own intermediate steps can favor its own outputs, letting errors propagate into later reasoning. A collaborative learning scheme replaces Qwen3-14B with three heterogeneous models of 1.7B, 4B and 8B under a 13.7B total parameter budget, using explicit task-state handoffs so different parameterizations reassess prior reasoning, then trains them with protocol distillation and step-level supervision to decide when to continue, hand off or terminate. Across ten metrics on five benchmarks, the heterogeneous system beats the original 14B model on all ten, with an average absolute gain of 2.38%, also surpassing a 24B homogeneous multi-agent baseline while using fewer model invocations than multi-agent frameworks.

A single large model that both generates and evaluates its own intermediate steps can favor its own outputs, letting errors propagate into later reasoning. A collaborative learning scheme replaces Qwen3-14B with three heterogeneous models of 1.7B, 4B and 8B under a 13.7B total parameter budget, using explicit task-state handoffs so different parameterizations reassess prior reasoning, then trains them with protocol distillation and step-level supervision to decide when to continue, hand off or terminate. Across ten metrics on five benchmarks, the heterogeneous system beats the original 14B model on all ten, with an average absolute gain of 2.38%, also surpassing a 24B homogeneous multi-agent baseline while using fewer model invocations than multi-agent frameworks.