Inside a protein language model, binary patterns form a jellyfish: hydrophobic content sets the main axis while heptad phase curls into a ring
In ESM2 residue embeddings, polar/nonpolar binary patterns organize into a readable geometry. Generating 50 sequences for each of all 128 heptad binary patterns, 6400 in total, and running position-preserving PCA yields a jellyfish-shaped manifold: hydrophobic content spreads along PC1, while PC2 and PC3 encode heptad phase as near sine and cosine. A charge alphabet stays ordered, but scrambled amino-acid groupings or the non-protein model ByT5 lose the cyclic organization.