PALM learns overlapping domain and site labels together, reaching 0.941 domain-occurrence F1 on 2.3 million held-out proteins
Synopsis
Protein sequence databases are growing far faster than curated annotation, and existing language models learn only from sequence likelihood, so InterPro domain and functional-site knowledge stays indirect. PALM builds on ESM2 with LoRA adaptation, learned layer aggregation and a bidirectional LSTM encoder, and uses a soft-target loss that lets one residue keep several valid labels. On 2,309,606 held-out proteins it reached domain- and site-occurrence F1 of 0.941 and 0.877, and still 0.857 and 0.828 in the subset below 20% sequence identity.
Interpretation
Protein annotation moves from a single label to overlap-aware multilabel prediction: one residue can belong to a domain and several functional sites at once, and the model separately decides whether annotations occur and which InterPro classes they are. Earlier deep annotation methods usually collapse each residue to one final class, or emit a single non-overlapping domain segmentation, so nested domains and coincident site types cannot both be expressed. On 2,309,606 held-out proteins, domain occurrence reached 94.49% precision, 93.77% recall and 94.13% F1, site occurrence reached 87.74% F1, and relaxed top-class accuracy was 97.14% for domain type and 99.45% for site type.
Annotation supervision changes the protein representation itself, so the same embeddings carry information beyond sequence likelihood for pathogenicity calls and family organization. Frozen ESM2 embedding differences reached AUROC 0.710 on ClinVar missense variants, while PALM saw no variant-database labels during training. On 151,823 ClinVar missense variants, PALM embedding disruption alone reached AUROC 0.835, and an equal-weight combination with the ESM2 likelihood ratio reached 0.877; on CATH-Gene3D superfamily assignment at k=5, PALM scored 0.955 accuracy, ESM3 0.951 and ESM2 0.825.
The architecture separates sequence encoding from annotation decoding: the 30 layers of ESM2-150M pass through LoRA adapters and a learned scalar mixture into a bidirectional LSTM, and four heads then predict domain occurrence, site occurrence, domain class and site class. The training objective is a soft-target loss that spreads target probability equally over all valid classes at a residue, instead of rewarding only one correct label. Training used 15,365,469 UniRef50 sequences mapped to InterPro annotations, spanning 17,951 domain classes and 1,383 site classes; the sequence-identity stratification used 686,370 test proteins with completed alignments.
Perspective
Evaluation covers 2,309,606 held-out InterPro proteins and the 686,370 of them stratified by sequence identity; downstream tasks are limited to ClinVar, PubMind, CATH-Gene3D, Reactome and experimental screens such as FlexiNR and PTEN.
InterPro mixes curated knowledge with computationally inferred signatures, so incomplete annotations and correlated classes propagate into the model; CATH-Gene3D is related to the InterPro annotations used in training, so that superfamily benchmark is not fully independent; ClinVar is a submission-based database and PubMind is extracted from literature by large language models, each with its own coverage and evidence-quality limits.