Skip to main content

# SkillGym

2 related articles

Subscribe to updates

arXiv Oct 1, 2026
SkillGym turns 184k community skills into verifiable training environments, and its 9B fine-tuned model beats a 397B untrained model on two benchmarks
SkillGym proposes an automatic pipeline that crawls skills from the internet and keeps those whose workflows run reproducibly offline, uses a builder-reviewer agent system to construct skill-critical tasks across four reasoning structures each with a reference solution and an executable verifier, then collects successful trajectories from three teacher models and four agent harnesses for supervised finetuning, improving six models from 2B to 122B parameters on 22 of 24 comparisons across four skill-use benchmarks, with the 9B model outperforming Qwen3.5-397B-A17B on the SkillGym test set and SkillEval and raising the rate of reading the relevant skill from 28% to 96%.
SkillGym proposes an automatic pipeline that crawls skills from the internet and keeps those whose workflows run reproducibly offline, uses a builder-reviewer agent system to construct skill-critical tasks across four reasoning structures each with a reference solution and an executable verifier, then collects successful trajectories from three teacher models and four agent harnesses for supervised finetuning, improving six models from 2B to 122B parameters on 22 of 24 comparisons across four skill-use benchmarks, with the 9B model outperforming Qwen3.5-397B-A17B on the SkillGym test set and SkillEval and raising the rate of reading the relevant skill from 28% to 96%.
Renxi Wang（First author）
SkillGym Agent Skills Task Synthesis Supervised Finetuning LLM Agents
arXiv Sep 27, 2026
SkillGym turns human skills into verifiable training environments, letting a 35B model reach 51.47% on SkillsBench and exceed reported scores of Claude Sonnet 4.6 and others
SkillGym introduces a framework that transforms human-written agent skills into executable, verifiable training environments: its skill-to-task pipeline instantiates concrete tasks, verifies outcomes with code-based checkers, and assesses empirical skill dependence through contrastive executions; it constructs and releases 2,756 environments across 12 categories and collects 8,364 successful trajectories (averaging 49 tool calls and over 60k logged text tokens), supporting supervised fine-tuning on verified workflows and reinforcement learning with outcome-based rewards; under Claude Code, supervised fine-tuning improves Qwen3.5-35B-A3B by 199 Elo on GDPval-AA v2, 19.10 percentage points on Terminal-Bench 2.1, and 28.13 and 12.38 points on SkillsBench v1.
SkillGym introduces a framework that transforms human-written agent skills into executable, verifiable training environments: its skill-to-task pipeline instantiates concrete tasks, verifies outcomes with code-based checkers, and assesses empirical skill dependence through contrastive executions; it constructs and releases 2,756 environments across 12 categories and collects 8,364 successful trajectories (averaging 49 tool calls and over 60k logged text tokens), supporting supervised fine-tuning on verified workflows and reinforcement learning with outcome-based rewards; under Claude Code, supervised fine-tuning improves Qwen3.5-35B-A3B by 199 Elo on GDPval-AA v2, 19.10 percentage points on Terminal-Bench 2.1, and 28.13 and 12.38 points on SkillsBench v1.
East China Normal University
Zhilong Ge（First author）
SkillGym Agent Skills Supervised Fine-tuning Reinforcement Learning large language model

You’re all caught up