IDiom and RL-SAE: an autoregressive model trained on 54 million predicted IDRs generates composable intrinsically disordered regions via sparse-autoencoder feature reinforcement
Related research and updates
Synopsis
The authors present IDiom, an autoregressive protein language model trained on IDiom-DB, a dataset of 54 million predicted intrinsically disordered regions (IDRs) curated from the AlphaFold Database, together with reinforcement learning with sparse autoencoder features (RL-SAE), a post-training method that rewards generation of sequences activating specified feature sets to control function-associated sequence patterns; across eight IDR design tasks, RL-SAE sequences activate on average 90% of 30 targeted features versus 24% for activation steering, improve predicted subcellular localization and transcriptional activity over steering and supervised fine-tuning, and allow features associated with distinct biological functions to be combined within individual sequences.
Figure 1: Training on 54M curated IDRs enables the generation of diverse and biologically plausible IDRs. (a) IDiom-DB curation workflow. (b) AlphaFold2 pLDDT distribution of curated IDRs and non-IDRs. (c) Fraction of curated training IDRs and experimental DisProt IDRs which are located at the N-terminus, C-terminus, and internally. (d) IDiom model perplexity on DisProt, validation, and generated IDRs. Solid and hatched bars indicate unprompted and prompted IDRs, respectively. (e) Amino acid compositional enrichment for training, generated, and DisProt IDRs, relative to the folded CATH sequences’ compositions. This panel’s legend also applies to panels (f) and (g). (f) Sequence identity of generated IDRs relative to training set IDRs, calculated with MMseqs2. (g) Predicted disorder scores of training, generated, and DisProt IDRs, alongside CATH sequences (higher is more disordered).
arXivInterpretation
IDiom is an autoregressive protein language model trained on IDiom-DB, a dataset of 54 million predicted IDRs curated from the AlphaFold Database, and it generates diverse sequences that recapitulate the composition, patterning, motifs, and predicted disorder of natural IDRs. Existing protein language models are trained on full-length sequences and thus learn a prior biased toward folded domains, and structure-based design methods do not readily apply to IDRs; restricting training data to predicted IDRs aligns the generative prior with disordered-region distributions. The abstract reports the training-data scale (54 million predicted IDRs) and the recapitulation of composition, patterning, motifs, and predicted disorder, which is model-level descriptive evidence.
RL-SAE is a post-training method that rewards generation of sequences activating specified feature sets, giving explicit control over function-associated sequence patterns. Compared with prior control approaches such as activation steering, RL-SAE treats sparse autoencoder features as interpretable design targets rather than relying on latent directions or supervised fine-tuning. Across eight IDR design tasks, RL-SAE sequences activate on average 90% of 30 targeted features versus 24% for activation steering, a quantified average comparison across tasks.
RL-SAE-generated IDRs improve predicted subcellular localization and transcriptional activity over steering and supervised fine-tuning, and enable features associated with distinct biological functions to be combined within individual sequences. Earlier control methods did not jointly show gains on function-related predicted metrics and multi-feature composability; this work treats interpretable features as stackable design units. Evidence consists of predicted-level metrics (predicted subcellular localization, predicted transcriptional activity) and feature activation proportions, i.e., computational evaluation rather than experimental validation.
The authors position IDiom and RL-SAE together as enabling interpretable and composable IDR design, and suggest RL-SAE could extend to other protein design settings where interpretable features provide useful design targets. Sparse autoencoder features are repurposed from an interpretability tool into a reinforcement-learning reward target, offering an explicit, composable control interface for disordered-region design. The extension claim is the authors' outlook based on this work's results; the abstract provides no empirical evidence for tasks beyond IDRs.
Perspective
The work targets researchers and protein engineers who need to design intrinsically disordered regions, in settings where the goal is to generate IDRs directed by function-associated sequence features and to combine multiple functional features within a single sequence. Methodologically, IDiom relies on predicted IDR data curated from the AlphaFold Database, and RL-SAE relies on sparse autoencoder features as reward signals, so both the granularity of control and its interpretability are premised on how those features are defined. The authors further propose that RL-SAE could extend to other protein design settings where interpretable features provide useful design targets, pointing toward transferring this post-training paradigm to different design tasks.
The abstract reports prediction-level results: feature activation proportions, predicted subcellular localization, and predicted transcriptional activity, and it remains unclear how closely these correspond to experimental measurements. RL-SAE's effects are evaluated over 30 targeted features and eight design tasks, so whether patterns outside that feature set are similarly controlled remains to be seen. IDiom-DB consists of predicted IDRs, and the influence of differences between training data and natural IDR distributions on generated outputs also warrants attention. In addition, this reading covers the abstract only; figures, baseline settings, and implementation details in the full text are not included, so the above judgments are limited to what the abstract states.