Skill libraries stop only growing: SkillForge retires and rewrites skills as the policy evolves, lifting WebShop success by 7.8% relative

An agent's skill library no longer only grows: each skill moves among trial, active, stable, and retired states according to its fitness, and skills evolve together with the policy during training. Earlier skill-augmented reinforcement learning treated the library as an append-only store, where an entry persisted once added and could not be corrected as the policy improved. Across three interactive agent benchmarks, ALFWorld, WebShop, and Search-Augmented QA, SkillForge reaches the highest aggregate success rate, up to 7.8% relative over the strongest baseline, with 92.4% on ALFWorld and 78.4% on WebShop.

Each skill tracks how often it was retrieved and how often those episodes succeeded, and that fitness decides promotion, demotion, retirement, or rewriting. Fitness is a cumulative running average with a minimum-usage gate, so a skill is not judged before enough evidence accumulates. A forging cycle runs every 10 training steps, rewriting up to 5 skills and retiring up to 3 per cycle under a total skill cap; rewriting uses the Kimi-K2.5 teacher model, and training runs 200 steps on 64 NVIDIA H200 GPUs in the verl framework.

Rewriting targets active skills whose fitness sits in a moderate band and whose usage is high enough, generating improved variants from their recent failure trajectories. Parent skills are sampled with inverse-fitness weighting, biasing selection toward skills with more room to improve rather than concentrating on a single outlier. The three environments generated 213 mutant skills with survival rates of 64.8%, 52.7%, and 62.9%; 46.4% of final non-retired skills are mutation children rather than seed skills.

The report covers SkillForge's training and evaluation on three interactive benchmarks, ALFWorld, WebShop, and Search-Augmented QA, with Qwen2.5-7B-Instruct as the base model, 200 training steps, and evaluation over five runs of the best validation checkpoint. It suits readers working on agent memory, skill-library management, and agentic reinforcement learning. The source leaves scalability to larger models as future work.

Fitness is computed from an episode-level shared reward, so when several skills are co-retrieved in one episode their individual contributions cannot be separated; the source reports that co-retrieval frequency stays below a stated fraction in each environment. All experiments use a 7B base model, and lifecycle dynamics under larger models remain unclear. Mutation depends on an external teacher model's instruction-following ability, and a weak teacher produces low-quality mutations that the trial gate must filter.