# Teacher Self-Deviation

The work argues that the token-level teacher–student discrepancy used by on-policy distillation (OPD) mixes in context-induced teacher-side variation, termed Teacher Self-Deviation (TSD), and introduces Cal-OPD, which estimates the teacher's self-deviation region via positive and negative privileged interventions and keeps only the residual beyond that region as the optimization signal, consistently outperforming standard OPD and its variants on mathematical reasoning benchmarks while retaining only about 52–65% of the original discrepancy.

The work argues that the token-level teacher–student discrepancy used by on-policy distillation (OPD) mixes in context-induced teacher-side variation, termed Teacher Self-Deviation (TSD), and introduces Cal-OPD, which estimates the teacher's self-deviation region via positive and negative privileged interventions and keeps only the residual beyond that region as the optimization signal, consistently outperforming standard OPD and its variants on mathematical reasoning benchmarks while retaining only about 52–65% of the original discrepancy.