A phage DNA model learns to denoise both ways: CPDD corrects 42.4% of visible wrong bases in one pass
Synopsis
Autoregressive DNA language models generate left to right, yet genome repair and editing usually condition on both flanks. The CreatiPhage team converted a 144.5M-parameter phage autoregressive model into CPDD, a 147.5M-parameter bidirectional discrete diffusion model; on 522 screened phage coding sequences it restores randomly masked bases more accurately than the comparable diffusion model PhaGen, and in a mixed-corruption test a single forward pass returns 42.4% of visible incorrect bases to the correct base.
Interpretation
Phage genome modeling gains a route that keeps a known scaffold and rebuilds only the uncertain regions: CPDD fills masked positions and rewrites visible wrong bases given the surrounding context, instead of generating from the start of the sequence. Autoregressive DNA language models generate left to right, and bidirectional encoders learn representations without generating directly, so region reconstruction conditioned on both flanks had no matching tool. On the screened panel of 522 coding sequences in 70 Mash clusters, single-pass restoration accuracy for CPDD and PhaGen was 0.470 and 0.451 at 5% masking and 0.406 and 0.392 at 50%, both above a rule using only codon-position frequencies (0.308–0.337).
The model treats visible bases as evidence rather than fixed truth, so it can correct positions already written incorrectly in the input. A diffusion model trained only with masking has no dedicated training signal for visible substitutions, so visible-error correction is an adapted stress test for it. Under a mixed-corruption protocol with 30% of positions corrupted and 10% of those left as visible substitutions, CPDD and PhaGen restored 42.4% and 22.7% of visible errors; with all corruptions visible the rates were 33.8% and 18.8%, and the fractions of visible errors left unchanged were 20.7% for CPDD and 52.0% for PhaGen.
The conversion keeps the pretrained backbone and swaps the causal structure for a bidirectional one, while letting the memory module handle the mask symbol. Training a diffusion language model from scratch means relearning sequence features, whereas converting an autoregressive checkpoint reuses pretrained weights. CPDD is initialized from the six-epoch CreatiPhage checkpoint and retains the 13-layer Transformer, Block Attention Residuals and Engram conditional memory, adding non-causal attention, a base-5 Engram with MASK as a fifth index, centered depthwise convolution and position-aligned nucleotide readout, growing from 144.5M to 147.5M parameters.
Perspective
Evaluation covers phage coding sequences: a main panel of 522 coding sequences, a 100-module functional panel, 24 source representatives and a 32-coding-sequence copy test, all computational, with no experimentally measured biological function. The interface restores fixed-length targets within a preserved scaffold, and insertions, deletions and coordinated whole-genome design are listed as future directions.
Long-gap recovery approaches codon-usage statistics: with whole coding sequences masked, CPDD reaches 0.361, inside the 0.264–0.372 range of rules using only base or codon-position frequencies and far from the 0.771–0.868 reachable when the encoded protein is known, so nucleotide agreement alone does not establish recovery of the original gene. Placing a verbatim copy of the coding sequence elsewhere in the same input gave no consistent recovery advantage across spacings of 50 bp to 16 kb; after 400 supervised updates on eight copy-bearing contexts, held-out recovery stayed around 0.15 with intact and shuffled copies performing similarly, and CPDD's final residue-weighted paired difference was 0.005 (95% interval −0.014 to 0.019). The evaluation panels were screened against documented training ranges but are enriched for Pseudomonas aeruginosa-related phages, distant homology cannot be excluded, and CPDD is not guaranteed reverse-complement equivariant.