# Meta-Learning

Meta-learning, which exposes a network to many related functions so it can later learn similar ones quickly, leaves the adapted parameter vectors of that network in an additive geometric relationship. On the ABgeom sequence-generation task, randomly initialized transformers reached a mean F-score of 0.062 after five training batches, while meta-learned initializations reached 1.00 after one step; parameter vectors recombined by symbolic analogy produced networks that implemented the intended functions without direct training, at a mean F-score of 1.00 over 100 analogies. The effect appeared across transformers and LSTMs and three function domains, but the analogies hold only around a single initialization.

Meta-learning, which exposes a network to many related functions so it can later learn similar ones quickly, leaves the adapted parameter vectors of that network in an additive geometric relationship. On the ABgeom sequence-generation task, randomly initialized transformers reached a mean F-score of 0.062 after five training batches, while meta-learned initializations reached 1.00 after one step; parameter vectors recombined by symbolic analogy produced networks that implemented the intended functions without direct training, at a mean F-score of 1.00 over 100 analogies. The effect appeared across transformers and LSTMs and three function domains, but the analogies hold only around a single initialization.

A meta model pre-trained offline on synthetic Wiener-Hammerstein systems predicts a new system's output zero-shot from a short input-output context, matches state-of-the-art accuracy on nonlinear identification benchmarks, and cuts an out-of-distribution continuous stirred tank reactor's prediction error from 1.249 to 0.242 by adapting only the input side.

A meta model pre-trained offline on synthetic Wiener-Hammerstein systems predicts a new system's output zero-shot from a short input-output context, matches state-of-the-art accuracy on nonlinear identification benchmarks, and cuts an out-of-distribution continuous stirred tank reactor's prediction error from 1.249 to 0.242 by adapting only the input side.

The work reports that directly incorporating a selection network into existing meta-learning for training-data selection (MTS) objectives causes unstable optimization and poor generalization due to weight suppression and persistent reliance on easy-to-learn features, and proposes Transferable Example Scoring and Selection (TESS), a scalable data-selection framework built on a Pointwise Value Matching (PVM) objective, which in experiments on LLM safety and targeted instruction tuning demonstrates strong transfer across datasets, from subsets to full corpora, and from smaller to larger models.

The work reports that directly incorporating a selection network into existing meta-learning for training-data selection (MTS) objectives causes unstable optimization and poor generalization due to weight suppression and persistent reliance on easy-to-learn features, and proposes Transferable Example Scoring and Selection (TESS), a scalable data-selection framework built on a Pointwise Value Matching (PVM) objective, which in experiments on LLM safety and targeted instruction tuning demonstrates strong transfer across datasets, from subsets to full corpora, and from smaller to larger models.