# Conditional Memory

Autoregressive DNA language models generate left to right, yet genome repair and editing usually condition on both flanks. The CreatiPhage team converted a 144.5M-parameter phage autoregressive model into CPDD, a 147.5M-parameter bidirectional discrete diffusion model; on 522 screened phage coding sequences it restores randomly masked bases more accurately than the comparable diffusion model PhaGen, and in a mixed-corruption test a single forward pass returns 42.4% of visible incorrect bases to the correct base.

Autoregressive DNA language models generate left to right, yet genome repair and editing usually condition on both flanks. The CreatiPhage team converted a 144.5M-parameter phage autoregressive model into CPDD, a 147.5M-parameter bidirectional discrete diffusion model; on 522 screened phage coding sequences it restores randomly masked bases more accurately than the comparable diffusion model PhaGen, and in a mixed-corruption test a single forward pass returns 42.4% of visible incorrect bases to the correct base.

Large language models store part of their factual knowledge in hash tables keyed by preceding token sequences, and low-bit quantization erases long-tail facts first, while giving more bits to frequently accessed rows does worse than choosing at random. GateRQ stores each row as a ladder of residual codes, computes each lookup's gate from a 1-bit code, and reads deeper levels only for the lookups the gate ranks highest. On DeepSeek-V4.1-Flash it cuts table memory from 23.6 to 3.04 GiB per GPU and raises vLLM throughput over FP8 tables in GPU memory by about 11% and about 31% on the two workloads, losing about 1 PopQA point at 1.50 bits per lookup.

Large language models store part of their factual knowledge in hash tables keyed by preceding token sequences, and low-bit quantization erases long-tail facts first, while giving more bits to frequently accessed rows does worse than choosing at random. GateRQ stores each row as a ladder of residual codes, computes each lookup's gate from a 1-bit code, and reads deeper levels only for the lookups the gate ranks highest. On DeepSeek-V4.1-Flash it cuts table memory from 23.6 to 3.04 GiB per GPU and raises vLLM throughput over FP8 tables in GPU memory by about 11% and about 31% on the two workloads, losing about 1 PopQA point at 1.50 bits per lookup.

EngramEdit proposes decoupled knowledge updates through conditional memory: it first computes target memory representations that make the model predict an updated fact across multiple expressions, then jointly solves updates to shared n-gram embeddings while penalizing changes to frequently reused embeddings more strongly; on LongCat-Flash-Lite, updating only conditional memory yields near-perfect editing success on CounterFact and ZsRE, revised facts transfer to unseen expressions and multi-hop reasoning, and unrelated knowledge and general capabilities are largely preserved.

EngramEdit proposes decoupled knowledge updates through conditional memory: it first computes target memory representations that make the model predict an updated fact across multiple expressions, then jointly solves updates to shared n-gram embeddings while penalizing changes to frequently reused embeddings more strongly; on LongCat-Flash-Lite, updating only conditional memory yields near-perfect editing success on CounterFact and ZsRE, revised facts transfer to unseen expressions and multi-hop reasoning, and unrelated knowledge and general capabilities are largely preserved.