Long-context regulatory models lose their edge across species: at fish distance 20 kb context turns negative while 1 kb local sequence still transfers
Synopsis
FunFormer compresses up to 20 kb of sequence into a fixed token set for binary regulatory-activity prediction, paired with an architecture- and parameter-matched control that reads only the central 1 kb, and both are transferred among six vertebrates. Within species the 20 kb version has the highest mean AUROC in all six genomes, 1.0 to 3.8 percentage points above three published 1 kb baselines; across species that context gain decays with divergence, nears zero at the amniote split, and reverses between 320 and 435 Mya, while the 1 kb model still transfers into fish.
Interpretation
The extra accuracy a long-context regulatory sequence model carries decays with evolutionary distance during cross-species transfer and turns negative beyond the tetrapod-fish split, while a model reading only the local 1 kb still transfers into fish genomes. Earlier cross-species transfer work established that absolute accuracy falls with distance, but the models compared there also differed in architecture, output type and training data, so the difference could not be assigned to context length. Across 23 source-target directions, the margin of FunFormer-20k over the parameter-matched FunFormer-1k was +3.4 percentage points among mammals, +0.4 pp into chicken and −7.8 pp across the tetrapod-fish split; relabeling the six species onto the tree in all 720 ways gave an observed correlation of r = −0.764 with P = 0.017.
The model does read flanking sequence outside the central 1 kb, but at fish distance the target genome's own flank supplies no more than an arbitrary stretch from the genome the model was trained on. Replacing the 19 kb outside each test interval's central 1 kb with a real GC-matched sequence from the same genome cost 8.1 pp among mammals and 9.6 pp into chicken, showing that the particular flank carries information. When a mammalian model was applied to a fish genome, a source-genome flank scored +4.75 pp above a target-genome flank (95% CI +3.00 to +6.50), whereas among mammals that difference was +0.58 pp (−0.27 to +1.42).
In variant effect prediction, whether a foreign model can substitute for a native one depends on how well the target species is already served by its own data, not on phylogenetic distance alone. On the data-limited pig target, three of five foreign models were statistically indistinguishable from the pig-trained model: zebrafish −0.47 pp, cattle −0.72 pp and chicken −1.12 pp. On the data-rich human target every foreign model was clearly worse, from −3.61 pp (cattle) to −6.24 pp (zebrafish), with all five intervals excluding zero and at most two of 49 tissues won.
Perspective
The conclusions rest on six vertebrates, four divergence times and the average behavior of typical loci on held-out chromosomes, with transfer scored only on (assay, tissue) channels shared by source and target.
The boundary can only be placed between 320 and 435 Mya because no sampled species falls inside that interval; both fish carry the teleost genome duplication, which coincides with the tetrapod-fish split, so this design cannot separate it from divergence time.