RAGenome scales a retrieval-based genomic language model to 13,312 nucleotides, lifting gene-finding MCC from 0.45 to 0.60
The authors introduce RAGenome, the first retrieval-based genomic language model: it retrieves homologous sequences from a 100-vertebrate whole-genome alignment, discards the 73% of tokens that are gaps, and scales pretraining context to 13,312 nucleotides, 100 times longer than existing MSA-based gLMs; it raises gene-finding MCC from GPN-Star's 0.45 to 0.60 and reaches 0.87 AUROC on pathogenic-variant prioritization, unifying long-range and evolutionary signals with 168M parameters and 16,032 GPU hours.