Longhui Zhang

Tool-using language-model agents often need dozens of interactions before a single success-or-failure label arrives, leaving no way to tell which step went wrong. STEP-VERIFIER turns a success-labeled corpus of logged interactions into action-level comparisons: the difference in average final return between two continuations that share a common prefix labels which of two actions was better, and a Bradley-Terry ranking head converts those pairs into calibrated per-step scores. On Llama-3.1-8B-Instruct and Mistral-7B-Instruct backbones, it beats outcome-only GRPO on the average metric in all six benchmark-backbone combinations across GAIA, ToolBench and AgentBoard, and its per-action scores agree with human judgments on 81.5% of pairs.

Tool-using language-model agents often need dozens of interactions before a single success-or-failure label arrives, leaving no way to tell which step went wrong. STEP-VERIFIER turns a success-labeled corpus of logged interactions into action-level comparisons: the difference in average final return between two continuations that share a common prefix labels which of two actions was better, and a Bradley-Terry ranking head converts those pairs into calibrated per-step scores. On Llama-3.1-8B-Instruct and Mistral-7B-Instruct backbones, it beats outcome-only GRPO on the average metric in all six benchmark-backbone combinations across GAIA, ToolBench and AgentBoard, and its per-action scores agree with human judgments on 81.5% of pairs.