Long-context regulatory models lose their edge across species: at fish distance 20 kb context turns negative while 1 kb local sequence still transfers
FunFormer compresses up to 20 kb of sequence into a fixed token set for binary regulatory-activity prediction, paired with an architecture- and parameter-matched control that reads only the central 1 kb, and both are transferred among six vertebrates. Within species the 20 kb version has the highest mean AUROC in all six genomes, 1.0 to 3.8 percentage points above three published 1 kb baselines; across species that context gain decays with divergence, nears zero at the amniote split, and reverses between 320 and 435 Mya, while the 1 kb model still transfers into fish.