Meta-learned networks can add and subtract functions: recombined parameters implement a target function with no direct training

A meta-learned starting point leaves the parameter vectors reached by rapid adaptation in an additive geometric relationship, so recombining those vectors along symbolic analogies between functions yields new networks that implement the intended function. Such meta-learned networks were already known to learn new functions quickly, but what the initial parameters encode to make that possible was unclear; randomly initialized networks learn the same functions slowly. On the ABgeom sequence-generation task, randomly initialized transformers reached a mean F-score of 0.062 after five batches, while meta-learned initializations reached 1.00 after one step, both averaged over 400 intended functions and ten network instances.

The procedure flattens each adapted network's parameters into one vector, treats it as a representation of the function that network implements, and writes a vector sum or difference following a symbolic relation of the form A is to B as C is to D to construct a new network's parameters directly. The test borrows the additive-analogy idea from word vectors, where king minus man plus woman approximates queen, but moves the object from word representations up to the implementation of whole functions. On ABgeom, 100 randomly generated analogies that vary one lexical component produced recombined networks with a mean F-score of 1.00, averaged over ten meta-learned transformers; across transformer and LSTM combined with the ABgeom, ABparity and ABlength domains, recombined networks scored well above chance.

The geometric structure holds only around a single meta-learned initialization, and analogies written across two different initializations all failed. If the structure came from a global organization imposed by the architecture or the task, the same difference vector should transfer between initializations. Cross-initialization analogies had a mean F-score of 0, and different LSTM instances spontaneously used different syntactic representations for ABlength.

The conclusions rest on two architectures, transformers and LSTMs, and three synthetic sequence-generation domains, ABgeom, ABparity and ABlength, with ten meta-learned initializations per setting and analogy performance measured by an F-score over sampled sequences. The work suits researchers interested in meta-learning, few-shot adaptation and representational geometry; extending the method to richer function domains will require ways to discover parameter-space geometry without positing the functional relationships in advance.

Each analogy tests only one hypothesis about how a function domain is represented geometrically, so a failed analogy does not establish that no other geometric structure exists; the authors mitigate this with a mean-max F-score over several syntactic representations but acknowledge they cannot exhaust every representation. Whether the geometry plays a causal role in rapid learning still requires direct follow-up work.