VenusGenome reads microbial genomes as context: frozen representations lead nine of eleven primary metrics
Synopsis
A microbial gene's function depends on the protein it encodes and on where that gene sits and what it neighbors, and encoding proteins one at a time discards that organization. VenusGenome takes frozen protein embeddings as gene inputs and uses a 75% masking autoencoder that forces reconstruction from sparse genomic context, with attention that separates within-contig from cross-contig relationships and encodes relative strand orientation. Pretrained on more than 1.7 million microbial genome assemblies, the frozen encoder reaches the highest mean performance among evaluated models on nine of eleven primary metrics, with zero-shot PPI genome-macro Spearman correlation of 0.300 versus 0.201 for BacFormer.
Interpretation
Microbial genome representation learning can now write a gene's position, contig membership and relative strand orientation directly into attention, instead of feeding an ordered string of protein embeddings into a model. Earlier gene-context models gLM and BacFormer both used 15% masking and kept masked positions in BERT-style encoders during pretraining; each also carried some structural information, such as gene orientation in gLM and contig embeddings plus rotary positional encodings in BacFormer. At 512M scale and matched 15% masking, masked autoencoding reaches a zero-shot PPI genome-macro Spearman correlation of 0.282 versus 0.192 for BERT-style reconstruction; removing contig-aware attention from the 75%-masking model drops the correlation from 0.293 to 0.203.
Only visible genes enter the encoder, and a lightweight decoder restores the masked protein embeddings, separating contextual encoding from reconstruction. The case for high-ratio masking comes from a preliminary observation: a linear predictor using the two preceding and two following genes reduced reconstruction error by 15.4% relative to a constant mean predictor, against 8.9% using four random genes from the same contig, so neighboring genes supply readily accessible reconstruction cues. The pretraining corpus is over 1.7 million microbial genome assemblies from isolate and metagenomic sources, with up to 4,096 genes per example and frozen ESM2-35M protein embeddings as gene inputs; on 4,096-gene inputs, raising masking from 15% to 75% reduces measured forward-backward time by 65.2%.
One frozen representation serves both genome-level phenotype prediction and gene-level tasks, and it beats baselines that take protein embeddings as input on zero-shot ranking. Mean pooling of ESM2-35M embeddings, the model's own frozen inputs, is outperformed on all five gene-level metrics, showing that genome-level pretraining adds information beyond aggregating individual proteins. Evaluation covers six whole-genome phenotype tasks and five gene-level metrics spanning BGCs, essential genes, operons and PPI; the six phenotype tasks use species-disjoint splits, BGC and essential-gene prediction use genome-disjoint splits, and supervised results average five probe-training seeds.
Perspective
The demonstrated scope is microbial genomes: pretraining uses over 1.7 million isolate and metagenomic assemblies, and evaluation is limited to six whole-genome phenotype tasks plus gene-level tasks including BGCs, essential genes, operons and PPI, with all representation encoders frozen. The intended reader is a researcher who needs gene- or genome-level representations for downstream prediction; assemblies longer than 4,096 genes are encoded as independent overlapping windows, with no direct attention across windows.
The gene-fitness analysis is explicitly exploratory: it evaluates average effects across assayed experiments and does not resolve condition-specific responses, with a median AUROC of 0.673 against 0.631 for an exploratory Evo2 standalone CDS likelihood baseline. The model uses frozen protein embeddings, so contextual learning is constrained to information retained by the protein encoder; the zero-shot PPI gain from 512M to 1B is smaller than from 85M to 512M, and raising the masking ratio leaves the 85M result nearly unchanged.