Diffusion-LM agents skip the text step: one hidden message beats four exchanges by 7.0 points on QASC
Synopsis
Diffusion language model agents can pass compressed denoising states to each other without decoding text first. A sender packs its hidden state into four signed dense vectors, and the receiver keeps its own context and trajectory while denoising. On QASC, matched messages raise accuracy from 72.6% to 80.8%, a message from another question drops it to 46.8%, and one late transfer reaches 87.8%, 7.0 points above four recurrent exchanges.
Interpretation
Two diffusion language model agents that denoise independently can pass answer-relevant information through one compressed continuous message, without the receiver first reading text the sender decoded. Multi-agent systems usually let agents communicate only after decoding text, discarding the sequence of revisable continuous states produced during denoising. On HM-Synth with Tiny-A2D, a matched message lifts accuracy from 8.6% to 72.9%, while a message from another example returns it to 9.3%, a paired difference of +63.6 points; the Dream-7B dependency-spanning task gives 9.8%, 100.0% and 0.0%.
The receiver uses the specific content of the message, not merely the fact that a message arrived. An accuracy gain alone cannot separate content from a generic activation, extra compute, or a distributional cue. On QASC, replacing the message with one produced for another question lowers accuracy from 80.8% to 46.8%, a content effect of +34.0 points; in the controlled task, swapping the sender's fact changes the receiver's answer in 89.4% of examples and selects the substituted target in 72.9%.
When a message arrives changes the collaboration, and more rounds are not necessarily better. The intuition that more interaction is better predicts four recurrent exchanges should beat a single transfer. The Dream-7B dependency-spanning task needs a query and a reply, where four recurrent exchanges reach 100.0% but one late transfer reaches only 10.8%; on QASC, where each agent can read its own evidence first, one late transfer reaches 87.8%, 7.0 points above four recurrent exchanges.
Perspective
The controlled evidence spans two backbones, Tiny-A2D and Dream-7B, and the natural-question evidence comes from Dream-7B on QASC, evaluated on all 1,000 derived development examples. The channel is a supervised, task-specific adapter while the backbone and language-model head stay frozen. The authors propose context-partitioned multi-agent code generation as the next setting, where different agents see the specification, source files, tests or execution feedback, and matched-versus-deranged interventions test whether code changes causally depend on the transmitted content.
Natural-question evidence currently comes only from Dream-7B on QASC, and larger agent teams, other diffusion backbones and long-form generation may produce different channel geometries. The channel relies on supervised task-specific adapters, so the results establish learned hidden communication rather than an emergent language or a task-independent protocol; initial zero-shot reuse across heterogeneous tasks yielded no detectable benefit. Matched text-channel and wall-clock comparisons remain future work, and dense states may encode sensitive information even when not human-readable, so the channel is not a privacy mechanism.