Evo 2 shows in-context learning on five binary classification tasks with F1 up to 0.902 on short sequences, but collapses at kilobase scale and the 7B model beats the 40B
This study maps the in-context learning operating regime of Evo 2, a nucleotide-level foundation genomic language model, across five binary classification tasks spanning biological and artificial sequences, finding robust performance on shorter natural sequences (F1=0.902 for miRNA, 0.785 for Toxins), degradation with sequence length and collapse at kilobase scale, no benefit from model scaling (the 7B model systematically outperforms the 40B variant), poor prediction of accuracy by perplexity, and mechanistic interpretability via logit-lens and Jacobian Scope suggesting a prediction-generalisation trade-off and that models might track prompt structure rather than signal-carrying content.