ProteinMPNN's latent space splits into readable axes for secondary structure, residue volume and heme coordination, and turning a branched-chain axis reshapes an antibody fragment's sequence
Synopsis
Sparse autoencoders expand the dense 128-dimensional latent space at each ProteinMPNN encoder layer into an overcomplete sparse space, where individual dimensions correlate with protein features such as secondary structure, residue volume, residue shape, disulfide bridges and ligand coordination. On the Brolucizumab short-chain variable fragment, the mod=0, 0.5 and 1 candidates with branched-chain-amino-acid dimensions raised held lower RMSD at 433 K than the native sequence, the ProteinMPNN sequence and the GeoEvoBuilder sequence, while mod=2, the largest raise, unfolded substantially at that temperature; the unmodified mod=0 candidate improved too, showing the autoencoder itself shifts predictions.
Interpretation
ProteinMPNN's latent space can now be split into individually inspectable dimensions instead of a dense vector that packs several features together. Previously these structure-to-sequence models compressed information into 128-dimensional node vectors, with multiple features sharing a dimension, so which protein properties drove a prediction could not be read off directly. After encodings from roughly 430,000 residues across 1,149 X-ray structures were randomly downsampled to 43,000 samples, SAE256 layer 3 reached a correlation of 0.893 with residue volume, 0.826 with beta strands and 0.802 with no structure, all above the original encodings without a sparse autoencoder.
A sparse autoencoder sits after each ProteinMPNN encoder layer, expanding dense activations into a wider, sparser representation and decoding back to the original width. ProteinMPNN weights stay frozen during training and only the sparse autoencoder is trained, so the expansion reads the model's existing representation rather than retraining a protein design model. Four models expanded the vector 1, 2, 4 and 8 times, to 128, 256, 512 and 1024 dimensions, each trained independently across the three encoder layers; SAE256 and SAE512 performed best on feature correlation, and SAE256 was chosen for later steering because it is smaller.
Sparse activations can be raised directly to change the sequences ProteinMPNN outputs, not merely to observe them. Correlation only shows a dimension co-occurs with a feature, while raising an activation tests whether that relationship affects prediction. Of Brolucizumab's 252 residues, 104 buried residues with solvent-accessible surface area below 20% had branched-chain-amino-acid dimensions raised, and the mod=0, 0.5 and 1 candidates held RMSD near 5 Å at 433 K while the native, ProteinMPNN and GeoEvoBuilder sequences unfolded substantially at that temperature, as did mod=2.
Perspective
The demonstrated scope is the node latent space of the ProteinMPNN encoder; edge latent space was not evaluated. Feature correlation used 1,149 X-ray structures with resolution better than 2.5 Å and fewer than 500 residues, and steering was tested on one short-chain variable fragment, Brolucizumab, with egg white cystatin 1CEW used to check sparse-autoencoder bias.
Correlations may not be causal, and the authors state the actual relationship can only be determined by future experiments. Qualitative feature annotation was done by hand without cross-validation across many users, and the authors say it is not definitive proof that features exist. The mod=0 candidate with no activation change also improved thermostability; the authors attribute this to sparse-autoencoder training residual shifting predictions toward more hydrophobic amino acids, and note that interactions between dimensions can weaken steering.