Salesforce (United States)

Agents often recognize that a task lacks a prerequisite yet keep acting; a new evaluation scores recognition and warranted stopping separately, and harness search without model-weight updates raises strict self-awareness from 20% to 60%.

Agents often recognize that a task lacks a prerequisite yet keep acting; a new evaluation scores recognition and warranted stopping separately, and harness search without model-weight updates raises strict self-awareness from 20% to 60%.

The work introduces CoTrace, a harness-aware data recipe for alternating model–harness co-evolution that explicitly governs trajectory routing, provenance matching, and curriculum refresh: execution failures are routed to harness synthesis while policy supervision uses only verified successes matched to the adopted runtime. On the 102-task Tmax promotion split, Qwen3.5-9B advances from 78 to 88 solved tasks under supervised fine-tuning and reaches 90 with an online reinforcement variant, whereas much larger corpora pooled across sibling harnesses yield no accepted model update.

The work introduces CoTrace, a harness-aware data recipe for alternating model–harness co-evolution that explicitly governs trajectory routing, provenance matching, and curriculum refresh: execution failures are routed to harness synthesis while policy supervision uses only verified successes matched to the adopted runtime. On the 102-task Tmax promotion split, Qwen3.5-9B advances from 78 to 88 solved tasks under supervised fine-tuning and reaches 90 with an online reinforcement variant, whereas much larger corpora pooled across sibling harnesses yield no accepted model update.