The distillation temperature everyone left at 1: raise it and forward KL overtakes reverse KL, adding 4.29 average ROUGE-L on GPT-2 Base
LLM distillation has largely kept the temperature at τ=1 and concluded that reverse KL beats forward KL. A gradient analysis shows temperature changes what forward KL learns but mainly rescales reverse KL gradients; across GPT-2 and OPT instruction-following benchmarks reverse KL leads at τ=1, forward KL overtakes at higher temperatures, and in on-policy GPT-2 Base distillation forward KL rises from 16.95 to 21.24 average ROUGE-L.