Cellular world models' high correlations fall apart: GEARS scores 0.977 globally but only 0.076 on perturbation deltas

Evaluation scores for cellular perturbation-response models are inflated by genes that do not respond: the same prediction looks near-perfect on global expression and near-irrelevant on the expression change the perturbation causes. These models have been judged almost exclusively by global Pearson correlation between predicted and observed transcriptomes, yet a typical CRISPR perturbation alters only tens to hundreds of the thousands of measured genes, and the rest match control. On 509 unseen single perturbations and 5000 genes from the Replogle K562 screen, GEARS reaches global Pearson 0.977 and delta Pearson 0.076; on 107 held-out conditions from Norman A549, global 0.877 corresponds to delta 0.548.

A baseline that ignores perturbation identity entirely beats the deep model in delta space, showing that the global score does not reflect perturbation-specific information. That baseline outputs the same mean training delta for every test perturbation, without distinguishing which perturbation it is. On Replogle the trivial baseline reaches delta Pearson 0.267, 3.5 times the 0.076 of GEARS; stratifying genes by response strength, absolute-expression correlation rises to about 0.98 in the most responsive decile while delta correlation stays between 0.03 and 0.25 in every decile.

Two post-hoc corrections that require no retraining pull predicted deltas toward the observed ones, and nearest-neighbor retrieval on predicted profiles is the more effective of the two. The corrections use only what the model already outputs: a random walk with restart on a co-expression graph built from control cells spreads the one-step delta along co-expressed neighborhoods, while cosine-similarity matching retrieves the most similar training perturbations and averages their true deltas. Random walk with restart raises delta Pearson from 0.076 to 0.103 across three seeds with standard deviation 0.004; nearest-neighbor matching with k=50 reaches 0.211 and top-20 differentially expressed gene Pearson 0.280, yet both remain below the trivial baseline's 0.267.

The residual sits mainly in the geometry of perturbation representations rather than in output decoder capacity, and writing delta alignment into the training objective recovers part of it. An oracle retrieval that uses true test deltas to find neighbors gives an upper bound of 0.592, a 2.8-fold distance from the 0.211 of predicted-delta retrieval that no decoder improvement can close. On the Replogle holdout, delta-aligned training at weight 3.0 raises delta Pearson to 0.179 with direction accuracy 0.680 and gene recovery 1.52, above the trivial baseline's 0.567 and 1.19; combined with nearest-neighbor matching it reaches 0.217 while global Pearson falls only from 0.977 to 0.966.

The conclusions rest on 509 held-out single perturbations from the Replogle K562 knockout screen and 107 held-out conditions from the Norman A549 overexpression screen, with 30 cells per condition to fit 8 GB of GPU memory; on the deep-model side only GEARS is evaluated, and newer models such as PRiMeFlow, scDFM and OCOO-T are not included.

The authors note that full-data experiments may change absolute numbers, though the downsampling analysis indicates relative rankings hold; the delta gap may vary across cell types and perturbation types, and amplitude calibration helps only in the large-gap Replogle regime while slightly lowering delta Pearson on Norman.