Transluce automates the hunt for training data with surprising side effects, yielding 6,575 datasets that mostly transfer across models
Synopsis
Researchers at Transluce and Berkeley turned datasets with surprising side effects from hand-built curiosities into something searchable at scale: given a target behavior, evolution, reinforcement learning and two subliminal-learning rewrites generate candidate data, and judges never used in the search filter it, producing 6,575 datasets across 8 behaviors. The data reads like ordinary post-training data yet makes a finetuned model show the target behavior; on 64 datasets, ML researchers predicted the side effect 63% of the time against 50% by chance.
Interpretation
Training data can shift a language model's behavior outside the training domain, and datasets with such surprising side effects previously numbered only in the dozens, mostly built by hand or found by accident. Prior work reported between a handful and 39 such datasets, drawn from emergent misalignment, weird generalization, subliminal learning, logit-linear selection and phantom transfer. The automated pipeline kept 6,575 of 9,119 candidate datasets across 8 behaviors, with 517 to 1,279 datasets per behavior, far beyond the dozens reported before.
The pipeline splits a side-effect dataset into three automatically scored conditions: the target behavior appears more often after finetuning than at base rate, the data reads like real post-training data, and a human auditor reading it would not anticipate the side effect. Earlier work did not write all three conditions as optimizable objectives, and known datasets carried no common naturalness or surprisingness scores. Stage one searches against a cheap proxy judge, gemma4-31b, and stage two re-scores and filters with claude-opus-4.8 and the gpt-5.6 family, which the search never touched, to limit reward hacking of the proxy.
Four methods supply candidates: evolutionary search over dataset specifications, a data generator trained with dataset policy gradient, and the subliminal-learning methods phantom transfer and logit-linear selection. Phantom transfer and logit-linear selection rewrite or filter an existing dataset once, while evolutionary search and dataset policy gradient optimize the data directly. On 64 datasets, ML researchers named the behavior a dataset induces in 63% of trials against 50% by chance, and gpt-5.6-sol reached 86% on the same set.
Most datasets keep working beyond the model used for search: side effects found on qwen3-1.7b reappear after finetuning five other models. Search optimized only against qwen3-1.7b, so whether the side effects were specific to that model was unknown. 72% of datasets transferred to all five held-out models, more than 90% transferred to the llama family, and 82% transferred to qwen3-14b, eight times larger; by behavior, 85% transferred to all models for nineteenth_century but only 55% for drunk_driving.
Perspective
The demonstrated scope is 8 single-turn behaviors, all synthetic data, roughly 150 rows per dataset, with search optimized only on qwen3-1.7b; the authors note these datasets are much smaller than production-level data, so a theory of generalization that explains them may not transfer to in-the-wild side-effect datasets.
The authors note the objectives rely on LLM judges, so scores depend on the specific models and vary across resampling, and updating a judge requires re-scoring the whole collection; elicitation rates sit above base rate but are often modest in absolute terms, and some datasets do not reproduce their side effect on retraining; most datasets induce several behaviors, while the surprisingness metrics count only the target behavior as correct.