Masked diffusion post-training stops sampling blocks at random: a full-block estimator scores highest on four of five benchmarks
Post-training for masked diffusion language models relies on Monte Carlo estimates of a blockwise surrogate objective, and the existing scheme samples the block index at random, so one block may be evaluated repeatedly while others are never evaluated. A full-block estimator evaluates every block exactly once and averages the results; analysis shows it stays unbiased and removes the between-group variance from block selection. Across five benchmarks in mathematics, planning and coding it reaches the highest accuracy on four, averaging 12.79 points above the random block sampling baseline.