# Instruction Following

Instruction-following quality falls as load grows, but there was no load-denominated capacity with an uncertainty. A measurement instrument defines load as single-word replacement rules, making each application a program-checkable Bernoulli trial, then inverts a binned-then-smoothed statistical model to obtain capacity. For GPT-4o mini, 2958 requests and 642430 trials give a capacity of about 37–38 rules at a 0.95 share of right replacements and about 18–19 rules at full replacement density.

Instruction-following quality falls as load grows, but there was no load-denominated capacity with an uncertainty. A measurement instrument defines load as single-word replacement rules, making each application a program-checkable Bernoulli trial, then inverts a binned-then-smoothed statistical model to obtain capacity. For GPT-4o mini, 2958 requests and 642430 trials give a capacity of about 37–38 rules at a 0.95 share of right replacements and about 18–19 rules at full replacement density.

Verifiable instruction-following benchmarks check whether output constraints are met, not whether the response answers the request, so frontier models can earn full credit with little substantive content. Using the same verifier as a reinforcement-learning reward produces reward hacking: publicly released OLMo 3 checkpoints show higher constraint satisfaction but lower response quality. CaLIF uses two likelihoods from the frozen reference policy, one under the request and one under the request plus constraints, to define a band that pays a fixed bonus inside and zero above it, raising IFBench constraint satisfaction from 26.98 to 34.64 across five configurations from 1.7B to 8B while holding response quality at 4.59.

Verifiable instruction-following benchmarks check whether output constraints are met, not whether the response answers the request, so frontier models can earn full credit with little substantive content. Using the same verifier as a reinforcement-learning reward produces reward hacking: publicly released OLMo 3 checkpoints show higher constraint satisfaction but lower response quality. CaLIF uses two likelihoods from the frozen reference policy, one under the request and one under the request plus constraints, to define a band that pays a fixed bonus inside and zero above it, raising IFBench constraint satisfaction from 26.98 to 34.64 across five configurations from 1.7B to 8B while holding response quality at 4.59.

Language model agents are increasingly handed the post-training stack, yet a single agent converges on supervised fine-tuning and spends its effort reweighting data categories. AutoSFT splits the loop into an executor, four analysis subagents that read each round from training data, failures, training signals and history, and a synthesis subagent that writes new examples for undersupplied, verifier-checkable capabilities. Every round retrains from the same base checkpoint, so each component is attributable: analysis reaches 63.33 on IFEval at round six, above the single agent's best of 62.35 across 50 rounds.

Language model agents are increasingly handed the post-training stack, yet a single agent converges on supervised fine-tuning and spends its effort reweighting data categories. AutoSFT splits the loop into an executor, four analysis subagents that read each round from training data, failures, training signals and history, and a synthesis subagent that writes new examples for undersupplied, verifier-checkable capabilities. Every round retrains from the same base checkpoint, so each component is attributable: analysis reaches 63.33 on IFEval at round six, above the single agent's best of 62.35 across 50 rounds.

The work introduces RoboFollow, a diagnostic benchmark combining high scene entropy, an L0–L3 hierarchical perturbation protocol, and stage-wise Intent/Execution scoring, and finds across nine VLA and WAM policies that strong in-distribution L0 performance does not transfer to L1–L3, with stronger VLM backbones, QA co-training, LangForce, and classifier-free guidance all failing to close the gap.

The work introduces RoboFollow, a diagnostic benchmark combining high scene entropy, an L0–L3 hierarchical perturbation protocol, and stage-wise Intent/Execution scoring, and finds across nine VLA and WAM policies that strong in-distribution L0 performance does not transfer to L1–L3, with stronger VLM backbones, QA co-training, LangForce, and classifier-free guidance all failing to close the gap.

Starting from Llama 3.1 8B, the study first fine-tunes a reviewer on official ICLR reviews from 2018–2023, then trains four successor models on ICLR 2024 data with synthetic reviews generated by that reviewer mixed at 0%, 33%, 66%, and 100%, finding that higher synthetic exposure compresses rating distributions and monotonically reduces same-paper and corpus-level semantic diversity (about 11% and 5%), a pattern the authors call scientific-judgment collapse, and introduces TrustReviewer, which mitigates this tendency through training-time corpus curation and test-time paired activation steering.

Starting from Llama 3.1 8B, the study first fine-tunes a reviewer on official ICLR reviews from 2018–2023, then trains four successor models on ICLR 2024 data with synthetic reviews generated by that reviewer mixed at 0%, 33%, 66%, and 100%, finding that higher synthetic exposure compresses rating distributions and monotonically reduces same-paper and corpus-level semantic diversity (about 11% and 5%), a pattern the authors call scientific-judgment collapse, and introduces TrustReviewer, which mitigates this tendency through training-time corpus curation and test-time paired activation steering.