Protein language model steering drops the global mean: PULSE pairs similar backgrounds, with the largest predicted-score gain on toxicity
Synopsis
Protein language models can be steered toward a target property without changing their weights, but existing activation steering builds the direction from the difference between the global means of property-positive and property-negative proteins, ignoring that the useful direction shifts with sequence background. PULSE first matches positive and negative proteins by representation similarity, then aggregates similarity-weighted, unit-normalized differences between matched pairs into a steering vector. On solubility, immunogenicity and toxicity optimization, PULSE beats global centroid steering in both final predicted property scores and optimization success rates, with the largest score gain on toxicity and the largest success gain on immunogenicity.
Interpretation
Controllable protein sequence optimization gains a steering path that does not rely on a global average: the direction can be built per sequence background instead of shared across all targets from a population mean difference. The earlier ASPO method subtracted the mean representation of property-negative proteins from that of property-positive proteins to get one global steering vector, which preserves no individual correspondences and can obscure how property directions differ across sequence backgrounds. On solubility, immunogenicity and toxicity, PULSE's mean final predicted property scores exceed ASPO's, with paired differences of 0.00715, 0.01724 and 0.05417, all significant after Holm correction; optimization success is also higher on all three, with the largest score gain on toxicity and the largest success gain on immunogenicity.
The method first finds positive-negative protein pairs with similar backgrounds, then turns their representation differences into a steering direction. Pairing is measured by cosine similarity at a chosen layer and solved with the Hungarian algorithm to maximize total similarity over disjoint pairs, with the number of pairs set by the smaller class; pairing is computed once and reused at every layer. Each pair's positive-minus-negative difference is unit-normalized and aggregated with softmax weights from the pair similarity at temperature 0.10; the resulting vector is rescaled to the norm of the ASPO vector, so the two methods differ only in direction construction while sharing site selection and decoding.
Whether optimization is reliable depends on whether each realized representation shift follows property-improving directions from similar backgrounds, not on the static steering vector alone. Static diagnostics disagree with optimization performance: PULSE shows larger mean angular deviation than ASPO on all three properties, and Local Static Vector Support shows no significant difference between the two methods. Trajectory-level Local Directional Support (LDS), summarized by the lower quartile, is higher for PULSE on all three properties with paired 95% confidence intervals excluding zero, rising from 57.75% to 72.15% on toxicity; PULSE keeps the higher mean across 10 independently sampled reference pools.
Perspective
The findings rest on the ESMC-600M backbone and scores from external property predictors rather than wet-lab measurements; evaluation covers solubility, immunogenicity and toxicity with 100, 100 and 50 test target sequences, using 4 optimization steps, 2 mutation sites per step and deterministic decoding. LDS requires an independent labeled reference pool, and its neighborhood is fixed at the starting protein.
The association between local directional alignment and optimization success is not established as causal; evaluation covers a single model backbone and predictor-based outcomes, and broader model and optimization-budget comparisons plus wet-lab validation remain next steps.