Cell foundation models stop chasing alignment: a new score shows fusion pays off only when modalities truly complement
Synopsis
Multimodal cell foundation models usually freeze each modality's encoder and train only a fusion interface, yet whether fusion adds information beyond the best single modality has lacked a test. Researchers introduce the Synergistic Information Score (SIS), which measures accessible synergy as the gain of a fused representation over the strongest unimodal one under a linear probe. Across lung, thymus and breast spatial transcriptomics, unimodal baselines suffice in redundancy-dominated tasks, while non-spectral fusion reaches higher SIS in the resolution-mismatched thymus task.
Interpretation
Whether multimodal fusion is worth it can be judged by whether the fused representation beats the strongest unimodal representation under a fixed linear probe, not by a higher F1 or R2 alone. Standard downstream metrics report predictive accuracy but cannot separate gains from shared redundant information from gains from cross-modal complementary information. On three spatial transcriptomics datasets, lung, thymus and breast, frozen UNI-2 histopathology and Nicheformer gene-expression encoders were paired and ten fusion interfaces compared against unimodal baselines.
The approach writes fusion gain as a relative quantity: the fractional performance improvement of the fused representation over the strongest unimodal one, positive for accessible synergy and near zero or negative for redundancy dominance or fusion degradation. The quantity uses the minimum mutual information assumption from partial information decomposition, treating the weaker modality's information as redundant with the stronger one, which yields a conservative estimate. It is estimated with linear probes on frozen representations, F1 Macro for classification and R2 for regression, with bootstrap confidence intervals and paired Wilcoxon tests reported.
The value of fusion shifts with a task's information structure: simple concatenation is near the ceiling when local correspondence is tight, and interfaces that learn nonlinear interactions gain ground as spatial ambiguity grows. In the thymus data, Visium gene expression covers coarse spots spanning multiple cells and mismatches fine histology; there the non-spectral methods DCCA and CoMM reach SIS of 0.142 and 0.120, above concatenation at 0.072 and CCA at 0.057. On the lung dataset, SIS for neighbour cell-type composition regression rises from about 0 to above 0.15 as centre-to-neighbour distance grows, while SIS for neighbour niche classification stays comparatively flat.
Perspective
The conclusions rest on frozen unimodal encoders plus a learned fusion interface, evaluated with linear probes; the spectral-ceiling analysis is explicitly limited to linear fusion maps and frozen encoders, and the authors note that end-to-end training changes the geometry of cross-modal representations. Evaluation covers three spatial transcriptomics datasets, pulmonary fibrosis lung, developmental thymus and breast cancer, on niche classification, cell-type composition regression, spatial neighbour prediction and spatial and niche consistency.
SIS is probe-dependent and here measures linearly accessible synergy, which the authors describe as a diagnostic of a task's information regime rather than a prescriptive criterion. Under strong modality imbalance SIS is conservative, and a negative value means fusion does not beat the dominant expert, not that the weaker modality carries no biological information. The authors also note that fine rank differences among close fusion methods are not individually significant under the fivefold evaluation, and that multi-omics application remains a pilot exploration.