# Lexical Long-Tail Collapse

Continuous diffusion language models generate a whole sentence in parallel without drawing a word at intermediate steps, and the cost is a collapsed lexical tail: ELF nearly matches human text on common words, yet its rarest words appear at only a fraction of the human rate, a gap that persists from 105M to 652M parameters and under every sampler setting tested. GRAFT reads the decoding head mid-generation, commits the rare words the average is about to discard, and regenerates from noise, raising the rare-word share from 0.42–0.77 to 0.82–1.25 of the human rate across three ELF scales and the independent LangFlow implementation, at the cost of a small rise in non-word rate on some models.

Continuous diffusion language models generate a whole sentence in parallel without drawing a word at intermediate steps, and the cost is a collapsed lexical tail: ELF nearly matches human text on common words, yet its rarest words appear at only a fraction of the human rate, a gap that persists from 105M to 652M parameters and under every sampler setting tested. GRAFT reads the decoding head mid-generation, commits the rare words the average is about to discard, and regenerates from noise, raising the rare-word share from 0.42–0.77 to 0.82–1.25 of the human rate across three ELF scales and the independent LangFlow implementation, at the cost of a small rise in non-word rate on some models.