Optimization algorithms stop being rewritten each time: treating iteration trajectories as experience yields a model that transfers across problems and larger dimensions
Synopsis
Iterative optimization algorithms can be turned into trainable models instead of being repeatedly rewritten as programs. The approach treats algorithm execution trajectories as computational experience, models optimization autoregressively so that the preceding generation forms the computational context for the next population, and uses a dimension-independent population-temporal architecture to capture both population interactions and iterative dynamics. The model first acquires search knowledge by imitation from heterogeneous teacher algorithms, then improves itself directly through optimization feedback, consolidating heterogeneous search knowledge into a single model that transfers to unseen, substantially larger dimensions.
Interpretation
Iterative optimization algorithms are handled as trainable objects, with algorithm execution trajectories serving as training data, so a model learns how algorithmic states evolve over iterations and reproduces the computational behavior of existing algorithms. Previously algorithms were programs designed by hand and redesigned for each new problem. The shift is shown with iterative optimization as the paradigm's first realization: the optimization process is written as an autoregressive process in which the preceding generation forms the computational context for generating the next population.
The model uses a dimension-independent population-temporal architecture that expresses population interactions and cross-iteration dynamics within one structure. The architecture is paired with an imitation-to-surpassing learning procedure rather than only copying the behavior of a single algorithm. The procedure first acquires algorithmic knowledge from heterogeneous teacher algorithms and then improves the resulting optimizer directly through optimization feedback.
The trained optimizer consolidates search knowledge from different teacher algorithms into one model and gains stronger optimization capability through subsequent improvement. It is not confined to reproducing one specific algorithm but brings heterogeneous search knowledge into a single trainable model. The acquired knowledge transfers across problems and generalizes to substantially larger dimensions unseen during training.
Perspective
The demonstrated scope is iterative optimization as the paradigm's first realization, with validation centered on cross-problem transfer and generalization to larger unseen dimensions on optimization tasks. It suits researchers interested in optimization solver design and learned algorithms.
The material presents this direction as a promising path rather than an established general algorithm model, and the claim that algorithmic capability accumulates through learning rests on this first realization.