# LLM Post-Training

RSI Arena hands language-model post-training to ten frontier models: each agent starts from the same 30-billion-parameter base, picks its own data, writes its own training code and nominates checkpoints within six days and 1,000 GPU-hours. All ten produced working assistants, and the model trained by GPT-6 Astra closed a median 76% of the base-to-Instruct gap across 12 capability benchmarks; with answer length and formatting controlled, its human-preference rating is level with NVIDIA Instruct, 1,095 against 1,094. A second stage returned each model's blind battle records to its agent, and eight of the ten models improved their style-controlled rating relative to NVIDIA Instruct, while benchmark gains stayed modest and showed no clear link to preference gains.

RSI Arena hands language-model post-training to ten frontier models: each agent starts from the same 30-billion-parameter base, picks its own data, writes its own training code and nominates checkpoints within six days and 1,000 GPU-hours. All ten produced working assistants, and the model trained by GPT-6 Astra closed a median 76% of the base-to-Instruct gap across 12 capability benchmarks; with answer length and formatting controlled, its human-preference rating is level with NVIDIA Instruct, 1,095 against 1,094. A second stage returned each model's blind battle records to its agent, and eight of the ten models improved their style-controlled rating relative to NVIDIA Instruct, while benchmark gains stayed modest and showed no clear link to preference gains.

For small merchants whose catalogs fit inside a long-context window, RPTune couples an encoder–reorganizer learned catalog curator with LLM post-training using automatically generated, catalog-grounded supervision: across 7 real merchants and 100 complex conversational queries each, context curation raises exact-match accuracy by 13.5 points on average and up to 20 points across 9 frozen LLM backbones, curation alone adds 14.6 points on gemini-3.7-flash, and curation plus post-training lifts gemma-4-E4B-it from 10.7% to 31.0% EM.

For small merchants whose catalogs fit inside a long-context window, RPTune couples an encoder–reorganizer learned catalog curator with LLM post-training using automatically generated, catalog-grounded supervision: across 7 real merchants and 100 complex conversational queries each, context curation raises exact-match accuracy by 13.5 points on average and up to 20 points across 9 frozen LLM backbones, curation alone adds 14.6 points on gemini-3.7-flash, and curation plus post-training lifts gemma-4-E4B-it from 10.7% to 31.0% EM.

ROSS introduces a selective-supervision relearning procedure that takes historical self-generated rollouts saved from domain-specific RL, multi-teacher on-policy distillation (MOPD), or agentic RL, uses an outcome verifier to keep successful trajectories and an LLM reviewer to mark which model-generated spans are worth imitating, then applies loss only to those selected tokens while keeping the full trajectory as context, improving upstream checkpoints through an offline SFT stage without new policy rollouts and raising Qwen3.6-35B-A3B's six-benchmark MOPD average from 58.40% to 62.20% and SWE-bench Verified from 64.20% to 68.40%.

ROSS introduces a selective-supervision relearning procedure that takes historical self-generated rollouts saved from domain-specific RL, multi-teacher on-policy distillation (MOPD), or agentic RL, uses an outcome verifier to keep successful trajectories and an LLM reviewer to mark which model-generated spans are worth imitating, then applies loss only to those selected tokens while keeping the full trajectory as context, improving upstream checkpoints through an offline SFT stage without new policy rollouts and raising Qwen3.6-35B-A3B's six-benchmark MOPD average from 58.40% to 62.20% and SWE-bench Verified from 64.20% to 68.40%.