Huanxin Sheng

The work reframes token selection in on-policy distillation (OPD) as a gradient-estimation reliability problem at a fixed prefix, decomposes the one-sample reverse-KL gradient into signal and sampling noise in information geometry, proposes the information-efficiency ratio (IER) as a signal-to-noise ratio under the optimal scalar baseline, and approximates it on a top-K candidate set for ranking; on mathematical and medical reasoning, IER alone approaches or exceeds full OPD at token budgets of 0.1%–1%, and combined with existing usefulness scores (IER-OR, IER-AND) it matches or exceeds full OPD in multiple settings.

The work reframes token selection in on-policy distillation (OPD) as a gradient-estimation reliability problem at a fixed prefix, decomposes the one-sample reverse-KL gradient into signal and sampling noise in information geometry, proposes the information-efficiency ratio (IER) as a signal-to-noise ratio under the optimal scalar baseline, and approximates it on a top-K candidate set for ranking; on mathematical and medical reasoning, IER alone approaches or exceeds full OPD at token budgets of 0.1%–1%, and combined with existing usefulness scores (IER-OR, IER-AND) it matches or exceeds full OPD in multiple settings.