Discrete diffusion sampling stops rerolling at every jump: exponential residual flows make the trajectory deterministic after initialization and enable consistency distillation

Discrete diffusion models can now have a sampling trajectory that depends only on its initial randomness, instead of redrawing at every jump. The standard approach was Gillespie's algorithm or ancestral sampling of the discrete-time reverse chain, both of which draw fresh randomness at each jump, so the trajectory itself is stochastic. The construction is proved to preserve the marginals of the original continuous-time Markov chain at every time, with residuals remaining independent standard exponential variables independent of the discrete state at that time.

The method gives every position and token a nonnegative residual variable and changes the discrete state by one token only when some residual reaches zero. Exponential residual flows reuse the exponential variables of Gillespie's first-reaction method but stop resampling after a jump, letting all residuals evolve deterministically under piecewise ODEs set by the forward and reverse rate matrices. For learned models, sampling freezes the leave-one-out denoiser over each interval while keeping the known time dependence of the rates, with explicit forms for masked and uniform diffusion; the scheme is exact for factorized data when the denoiser is exact.

Deterministic trajectories give consistency distillation a pathwise target in the discrete setting. The absence of a discrete analogue of the probability-flow ODE had often been used as an argument against trajectory-based distillation. On TinyGSM, uniform-diffusion distillation doubles the accuracy of the strongest baseline at 2 and 4 model evaluations and is also higher at 8 and 16, approaching the teacher by 32; on OpenWebText, uniform-diffusion distillation has lower perplexity at matched entropy than the other baselines, while masked-diffusion distillation roughly matches the second-best baseline.

The demonstrated scope is masked and uniform diffusion on OpenWebText and TinyGSM, distillation experiments on Sudoku completion, and a comparison of DiffusionGemma on 140 MMLU-Pro questions; the sampling-quality conclusion is limited to matched model-evaluation budgets.

The deterministic trajectory generally differs in law from the reverse Markov chain and need not be Markovian; the largest distillation gains appear for uniform diffusion, while masked-diffusion results are only competitive with the compared baselines.