EvoGuard screens every online skill rewrite: 56.5% average success on ALFWorld versus 45.5% for accepting all updates
Synopsis
Language agents can store experience as natural-language skill documents, but rewriting a skill from a single trajectory after every deployed task lets its quality drift. EvoGuard turns each update into a two-way choice: offline, skills are scored on a private test set, rejection sampling keeps pairs with a clear utility gap, and a selector trained on the logits of two label tokens decides which skill to keep; at deployment, one forward pass settles which of the old and new skills stays. On ALFWorld, with Qwen3-4B and Qwen3-8B executors, it averages 56.5% success, against 45.5% for accepting every rewrite and 36.7% with no skill.
Interpretation
Online skill evolution gains a per-update acceptance decision: after each single-trajectory rewrite, the agent decides whether the new skill replaces the current one instead of adopting it by default. Earlier approaches either updated a skill offline from a batch of trajectories and adopted it directly, or kept it only after extra rollouts on a held-out validation set, which is too costly to run after every task. On ALFWorld, 42.2% of the 1,137 rewrites written by the base generator score below their parent skill on the private test set, with 7.7% tying and 50.0% improving; among the 900 rewrites from a reinforcement-learning-trained expert generator, 40.0% still score below their parent.
EvoGuard reduces comparing two skill texts to a single forward pass that generates no tokens and runs no extra environment rollouts. A held-out-validation check requires many extra rollouts for every update, costing far more than the task that triggered it. The selector uses a Qwen3-4B backbone with LoRA adapters only, reads the logits of two label tokens at the end of the prompt and renormalizes over them; on skill pairs unseen during training it picks the better skill 95.0% of the time.
Across three benchmarks, screened skill evolution beats accepting every rewrite and beats running without skills. Accepting every rewrite lets the skill drift with the task stream and even falls below no-skill execution on ScienceWorld. On ALFWorld the Qwen3-4B and Qwen3-8B executors average 56.5% success, against 45.5% for accept-all and 36.7% with no skill; ScienceWorld success rises from 10.2% to 15.3%; LiveCodeBench pass rate is 28.4%, 2.3 points above no skill.
Perspective
Experiments cover the text environments ALFWorld, ScienceWorld, LiveCodeBench and WebShop, with executors drawn from the Qwen3 model family; building and scoring the skill pool requires a one-time offline computation before deployment, and one selector is trained per benchmark.
The selector can only choose among the rewrites it receives and cannot repair task families where written skills do not help, falling below no-skill execution in some Pick2 and Heat settings; other model families may rank skills differently, and multimodal or embodied settings are left for future work.