Progressive networks stop piling up columns: decay the lateral gain to zero, prune the parent, and the outputs do not move
Synopsis
Progressive neural networks transfer knowledge by keeping every frozen earlier column, so stored state and inference grow with each generation. WeanNet keeps a single frozen parent, feeds its layer-wise activations to a fresh student through a scalar gain, and decays that gain linearly to zero during training, so the parent can be removed at deployment. On a concept-drift task it found the new correct action on drifted states more often than the architecturally matched PNN-Reset control at every tested entropy setting, though it also led on never-drifted states and stayed comparable to training from scratch.
Interpretation
The new method compresses progressive-style layer-wise feature transfer into a single retained parent, and the parent can be removed entirely after training, so deployed resident parameters no longer grow with generations. A full progressive network adds a column per generation and keeps all earlier columns, so stored parameters and the number of columns evaluated in one forward pass both grow with generations, and the lateral matrices can grow faster still. On the 24–64–64–8 concept-drift network, each agent holds 17,232 resident parameters during training and 6,280 after pruning the parent, a 63.6% reduction; both counts are independent of generation count, and the network evaluates two columns during training and one after removal.
The parent supplies layer-wise activations rather than output labels, so the student can use the task reward to decide which parent features to keep and which to ignore. Knowledge distillation treats the teacher's output distribution as an explicit training target, and that imitation objective can conflict with the new reward when the required behavior differs from the teacher's. Each student affine layer receives the matching frozen-parent activation through a trainable lateral matrix; an optional direct parent-logit pathway into the student output layer is enabled in the primary concept-drift and CartPole runs and is ablated on and off in each.
The lateral contribution is multiplied by a scalar gain that falls linearly with training progress, from 1.0 to 0 over the first 300 of 600 batches in the primary configuration. The architecturally matched PNN-Reset control keeps the same lateral matrices and injected activations but holds the gain at 1 throughout, so the two differ only in whether the gain decays. On concept drift the drift-found gap is positive at every entropy coefficient tested, at 0.213, 0.213, 0.215 and 0.190 for coefficients 0, 0.05, 0.1 and 0.2; the core-state gap is 0.116, which the prespecified confound check treats as mixed rather than cleanly drift-specific support.
Perspective
The demonstrated scope is layer-wise feature transfer from a single parent in multilayer-perceptron architectures, with exact parent removal once the gain reaches zero. The concept-drift task has 64 states, 8 actions and one decision per episode, with a 24–64–64–8 primary architecture and 32–32, 64–64 and 64–64–64 hidden-layer variants. CartPole serves as a five-generation gravity stress test, and the Roblox obstacle course as a five-seed descriptive training-reward comparison. The author proposes extending to longer task sequences, other control algorithms, convolutional or recurrent architectures, and testing the generation-handoff explanation.
The author states the results support bounded temporary feature transfer rather than universal performance superiority. On concept drift WeanNet also beats PNN-Reset on never-drifted core states, which the prespecified criterion treats as mixed support, so decay is not isolated as drift escape alone. The full progressive network is weak with the direct parent-logit pathway on and rises from 0.191 to 0.665 mean reward with it off, so its outcome is configuration-dependent. On CartPole, distillation and Regular have the highest mean returns, and paired intervals neither resolve nor establish equivalence among WeanNet, Reset and PNN-Reset. Roblox five-seed training rewards overlap, and the summaries do not measure entry into the newly opened region.