Gene combinations as an adversarial attack: GCG searches reprogramming factors in single-cell foundation models, beating greedy at equal budget
Synopsis
In single-cell foundation models, predicting the effect of overexpressing genes had been limited to one gene at a time or to enumerating a few hundred combinations. A Kyushu Institute of Technology team recast choosing which genes to overexpress as a discrete adversarial attack on gene tokens and searched combinations with Greedy Coordinate Gradient (GCG), a search built for language models. In Geneformer and scGPT, GCG raised similarity to the target cell state above gradient-pool greedy at equal forward-pass cost, with the margin growing with set size; measured against published reprogramming factors, however, ranking by single-gene effect recovered more of them.
Interpretation
The effect of a gene set cannot be obtained by adding up single-gene effects, so combinations have to be searched directly. In silico perturbation with single-cell foundation models had handled single genes or small user-specified sets, and combination effects were mainly evaluated by enumeration, which covers only a small range. Across 18 conversion pathways in Geneformer, the measured improvement of random sets regressed on the sum of member effects gave slopes of 0.491, 0.151 and 0.083 at k = 2, 4 and 8, with no pathway above 0.9 at any set size, where an additive objective would give 1.00.
Treating overexpressed genes as an adversarial suffix lets a language-model attack search the combination space for factors. GCG was developed to insert a few tokens after a prompt and steer a language model's output; here the inserted items are overexpressed genes in a cell's gene sequence. The objective is the cosine similarity between the perturbed cell embedding and the L2-normalized centroid of up to 200 target cells; each step takes the gradient with respect to the gene one-hot encoding, keeps the 256 top genes per position, samples 128 substitutions for exact forward-pass evaluation and accepts the best unconditionally, starting from a random set with three restarts per condition.
At equal forward-pass budget, GCG finds sets closer to the target state than greedy forward selection. Greedy adds the currently best single gene at each step and never revises members already chosen. Over 20 source cells per pathway for 18 pathways (360 pairs), the improvement ratio of GCG to gradient-pool greedy was 1.043, 1.181 and 1.399 in Geneformer at k = 2, 4 and 8 and 1.175, 1.190 and 1.204 in scGPT, with paired t-test p values below 2×10⁻³.
Scored against published reprogramming factors, searched sets hit less often than single-gene rankings, indicating that the model-induced similarity does not fully reflect gene regulatory logic. Ranking by single-gene effect is the standard use of in silico perturbation, and ranking by differential expression approximates how reprogramming factors have historically been nominated. In Geneformer at k = 8, the share of sets overlapping any single published protocol by at least one factor was 55.6% for differential-expression ranking, 44.4% for single-gene-effect ranking, 15.6% for GCG and 8.3% for gradient-pool greedy, against 1.45% for random 8-gene subsets; the two searches using the embedding gradient averaged 11.9% and the two not using it 4.4%.
Perspective
The demonstrated setting is two pretrained models, Geneformer and scGPT, 18 direct-reprogramming pathways from Tabula Sapiens, and a search objective defined as cosine similarity in embedding space; the non-additivity measurement and set-level hit rates were obtained in Geneformer only, and the comparison with greedy at its natural budget and the 2×2 decomposition used five pathways. The intended reader is a researcher working on reprogramming-factor selection or on evaluating single-cell foundation models. Experimental validation of the returned sets has not been done and is left as future work.
The authors note that the objective is embedding-space similarity rather than a measured biological outcome, and whether the returned eight-gene sets work requires experiment; the benchmark is partly circular, eight of the 18 pathways have three or fewer curated factors, and the pathway-specific AUC uses other pathways' factors as negatives, so absolute hit rates should be read against chance. The values also depend on one insertion convention in Geneformer, untuned batch and pool sizes, three restarts and a single random seed, and cost was counted in forward passes with a gradient charged as two, so wall-clock ratios may differ; Tabula Sapiens may partly overlap the models' pretraining data, a condition shared by all compared methods.