# DNA Foundation Models

3 related articles

Subscribe to updates

bioRxiv Oct 14, 2026
DNA-LLM2Vec turns causal DNA decoders into bidirectional encoders, with frozen-probe gains that shift by task and backbone
DNA-LLM2Vec converts pretrained causal DNA decoders into frozen bidirectional encoders in stages: it opens bidirectional context, repairs the resulting mismatch with masked next-token prediction, then adds contrastive views encoding strand and positional invariance. Across two backbones, DNAGPT and HyenaDNA, and seven biological task families, the benefit is not uniform but depends on task structure and backbone; crop contrastive learning helps most for extended-context families such as enhancer–promoter interaction and Virus-COVID, reaching frozen Avg-EPI MCC of 27.5% and 26.7%.
DNA-LLM2Vec converts pretrained causal DNA decoders into frozen bidirectional encoders in stages: it opens bidirectional context, repairs the resulting mismatch with masked next-token prediction, then adds contrastive views encoding strand and positional invariance. Across two backbones, DNAGPT and HyenaDNA, and seven biological task families, the benefit is not uniform but depends on task structure and backbone; crop contrastive learning helps most for extended-context families such as enhancer–promoter interaction and Virus-COVID, reaching frozen Avg-EPI MCC of 27.5% and 26.7%.
Queen's University Belfast Utrecht University Recursion (United States)
Yuan Liang（First author, Corresponding author）
DNA-LLM2Vec DNA Foundation Models Masked Next-Token Prediction Contrastive Learning Linear Probing
bioRxiv Nov 6, 2026
Flip the strand and DNA language models change their answer; MirrorDNA cuts both strands at mirrored points to give one prediction
DNA language models usually tokenize, train and test on one strand, so a sequence and its reverse complement can get different predictions. MirrorDNA uses a symmetric router that cuts both strands at mirrored positions and proves the whole model is equivariant to the reverse complement. On 12 strand-symmetric Nucleotide Transformer tasks it reaches an MCC of 0.491 on both strands, against 0.152, 0.187 and 0.080 for NT-v2, DNABERT-2 and HyenaDNA.
DNA language models usually tokenize, train and test on one strand, so a sequence and its reverse complement can get different predictions. MirrorDNA uses a symmetric router that cuts both strands at mirrored positions and proves the whole model is equivariant to the reverse complement. On 12 strand-symmetric Nucleotide Transformer tasks it reaches an MCC of 0.491 on both strands, against 0.152, 0.187 and 0.080 for NT-v2, DNABERT-2 and HyenaDNA.
BRAC University Bangladesh University of Engineering and Technology
Nirjhor Datta（First author）M. Sohel Rahman（Corresponding author）
MirrorDNA Reverse-Complement Equivariance DNA Foundation Models
bioRxiv Sep 21, 2026
EvSpark: Lossless Speculative Decoding for Hybrid DNA Foundation Models
EvSpark is a speculative decoding system for hybrid convolutional-recurrent-attention DNA foundation models such as Evo2: it verifies draft blocks in parallel and restores all three classes of inference state by selecting retained intermediate states without replay, reaching 2.96x on 43 real-sequence prompts and 3.27x including five synthetic controls on an Evo2 7B 48-prompt benchmark with three training seeds, retaining 1.84x-2.43x at 262k context, and yielding 2.18x-2.46x on real sequences and 2.51x-2.78x on the full suite for 20B and 40B targets.
EvSpark is a speculative decoding system for hybrid convolutional-recurrent-attention DNA foundation models such as Evo2: it verifies draft blocks in parallel and restores all three classes of inference state by selecting retained intermediate states without replay, reaching 2.96x on 43 real-sequence prompts and 3.27x including five synthetic controls on an Evo2 7B 48-prompt benchmark with three training seeds, retaining 1.84x-2.43x at 262k context, and yielding 2.18x-2.46x on real sequences and 2.51x-2.78x on the full suite for 20B and 40B targets.
Fudan University Zhongshan Hospital
Hao Ding（First author）Nannan Wu（Corresponding author）
EvSpark Speculative Decoding Evo2 DNA Foundation Models

You’re all caught up

DNA-LLM2Vec turns causal DNA decoders into bidirectional encoders, with frozen-probe gains that shift by task and backbone

Flip the strand and DNA language models change their answer; MirrorDNA cuts both strands at mirrored points to give one prediction

EvSpark: Lossless Speculative Decoding for Hybrid DNA Foundation Models