Single-cell regulation model Muss predicts expression for genes and peaks it never trained on, straight from DNA
Synopsis
Single-cell gene-regulation modeling has split into two paths: sequence models read DNA regulatory grammar but lack cell-level resolution, while single-cell omics models treat genes and peaks as fixed-vocabulary tokens and discard the sequence. Muss derives peak and gene representations from DNA with a frozen sequence encoder, links them through peak–peak self-attention and distance-biased peak–gene interactions, and trains with peak masked language modeling plus paired scATAC-seq and scRNA-seq. Across four paired datasets it still predicts expression for genes and peaks unseen in task training, reaching pseudobulk PCC 0.684 on PBMC10K in the unseen-sequence setting.
Interpretation
Single-cell gene-regulation prediction can now cover genes and peaks absent from task training instead of requiring them to sit in a fixed vocabulary. Earlier sequence models predict functional readouts within genomic windows, and single-cell omics models treat peaks and genes as fixed-vocabulary tokens, so neither gives sequence-grounded representations for previously unseen regulatory entities. On four paired scATAC–scRNA datasets (PBMC10K, BMMC, Openproblem, Bone), holding out one quarter of genomic regions under the Borzoi split, including 1,161 of the initial 5,257 genes, Muss reaches pseudobulk PCC 0.684 on PBMC10K in the unseen-sequence setting, above AlphaGenome at 0.609 and scooby at 0.533.
Peak and gene representations both come directly from DNA sequence, so the two scales share one sequence-defined semantic space. Fixed-vocabulary models treat each peak as an independent token, even when two peaks share sequence grammar or motif composition. Peak and gene sequences are encoded with the frozen AlphaGenome encoder at 128-bp resolution and average-pooled, then processed by one peak self-attention layer and two cross-scale interaction blocks, with cross-attention biased by Gaussian radial basis functions over genomic distance and reweighted by a monotonic distance decay.
The accessibility context learned by peak masked language modeling transfers beyond expression prediction. Frozen sequence embeddings alone do not capture cell-specific chromatin state. Zero-shot cell clustering on the held-out PBMC169K batch BIO_ddseq_1 gives Muss an ARI of 0.5377, against 0.4835 for ChromFound and 0.1032 for AlphaGenome; removing peak masked language modeling drops the ARI to 0.1221.
Perspective
The demonstrated scope is pseudobulk and single-cell expression prediction on four paired scATAC–scRNA datasets, enhancer–gene linking supported by GeneHancer with eQTL or Capture Hi-C evidence, motif-guided regulatory prioritization, and zero-shot clustering on the held-out PBMC169K batch; enhancer–gene evaluation covers only the training peak and gene vocabularies.
Motif presence does not establish transcription-factor binding or causal regulation, and motif-guided prioritization remains hypothesis-generating; baselines differ in how they encode genomic position, which affects that comparison. Exact peak self-attention has quadratic arithmetic cost in the number of peaks, and inference is slower than lightweight baselines, 61.18 ms per cell against 5.94 ms for ChromFound.