Same tools, different interface: agent behavior shifts, and repairing only the expression lifts six settings by up to 9.0 points
Synopsis
With executable capabilities held fixed, presenting them as MCP tools or as CLI commands still sends language agents down different task trajectories. Built from a shared semantic action contract, the comparison spans six models and three interactive environments, and 94.3% of 2,004 paired episodes show structured divergence. DART, which repairs only the interface expression once a semantic decision is validated, raises benchmark score in all six benchmark–interface settings by up to 9.0 points and lowers MCP token cost in all three environments.
Interpretation
The same set of executable operations, with only the model-facing interface changed, is enough to shift a language agent's task score and interaction cost. Earlier system-level MCP–CLI comparisons often changed tool granularity, prompting, validation or execution scaffolding at the same time, so a performance gap could not be attributed to representation alone. Across six models and the ALFWorld, WebShop and ScienceWorld environments, the size and direction of the interface effect vary by model–environment pair: Qwen3-14B trails CLI by 36.6 score points under MCP on ALFWorld (20.1 vs. 56.7) but exceeds it by 4.9 points on WebShop (64.4 vs. 59.5).
The interface difference goes beyond formatting and changes which semantic action the agent selects, where it fails, and how it recovers. Endpoint scores collapse distinct interaction pathways into one number, hiding differences in semantic choice, realization failure and interaction cost. On WebShop/Qwen3-14B, MCP shifts decoded action frequency by +6 percentage points toward click and −6 points from search; on WebShop/Qwen3-4B, retry acceptance is 50% under CLI and 77% under MCP, with later task success of 18% and 34%.
Divergence appears early in interaction and propagates through subsequent history, making the trajectory rather than the single response the right unit of analysis. Matched-state probes at fixed decision contexts locate where sensitivity enters but cannot show how it changes later observations, states and costs. Of 2,004 paired trajectories, 1,889 (94.3%) exhibit a structured divergence, the first divergence occurs near the beginning of interaction across most model–environment strata, and semantic-action differences account for the dominant share of divergent pairs.
Separating semantic decision from interface expression, and repairing only the expression when it fails, raises score and lowers token cost under MCP. An ordinary retry policy reselects a semantic action after a realization failure and can discard an already validated decision. For Gemini-2.5-Flash, DART gains +1.5/+9.0 points on ALFWorld, +0.8/+4.4 on WebShop and +6.7/+5.3 on ScienceWorld for CLI/MCP; under MCP it cuts tokens per task by 27.1k, 11.2k and 4.3k, while CLI changes are −0.8k, +4.8k and +1.6k.
Perspective
The findings cover paired evaluation of Qwen3-4B/8B/14B/32B, DeepSeek-v4-Flash and Gemini-2.5-Flash on ALFWorld, WebShop and ScienceWorld, while the paired DART comparison is run only on Gemini-2.5-Flash. It suits researchers and engineers working on tool-calling interface design and agent recovery policies.
The size and direction of the interface effect vary by model and environment, and no single direction of advantage holds across models; DART's score gains and MCP token reductions come from the six paired Gemini-2.5-Flash settings, and the intervention is not evaluated on the other models.