Same GitHub task, same right answers: the MCP server spends 59,417 input tokens where the gh CLI needs 1,421
Synopsis
OpenRouter compared three ways to give an agent the same GitHub capability, an MCP server, the gh command-line tool and a gh skill, counting input tokens across the whole task. On Claude Sonnet 5.5 all three answered correctly, but the GitHub MCP server with its 46 default tools used 59,417 input tokens, the CLI 1,421 and the skill 3,315. Scoping the server to the 2 tools the task needs cut it to 23,228, and capping tool results at 4,000 characters brought it to 5,357.
Interpretation
Wiring the same GitHub capability into an agent three different ways changes context cost by more than an order of magnitude, with no difference in correctness. An MCP server sends every tool's name, description and JSON schema on every request, while the CLI gives the model a shell tool and one line saying gh is installed and authenticated, and a skill gives a one-line index and loads its full SKILL.md only when a task needs it. On one GitHub task with Claude Sonnet 5.5, the CLI used 1,421 input tokens, the skill 3,315 and the MCP server with its 46 default tools 59,417, and every row in the table answered correctly; at Sonnet 5.5 prices the default MCP route cost $127 per 1,000 tasks against $7.50 for the CLI.
MCP's cost comes as much from raw tool results as from tool definitions, and those results are resent with every later request. get_latest_release returned GitHub's full release object, 42,182 characters, 87% of it describing 22 download files, while the task needed only the tag and date that gh release view returns with --json. That release object added about 20,400 tokens on Sonnet 5.5, more than all 46 tool definitions; scoping the server to two tools cut the first request from 19,487 to 1,406 tokens, yet the whole task still cost 23,228, and a 4,000-character cap brought it to 5,357 with every run correct.
Tool Search replaces up-front tool definitions with on-demand lookup, removing most of that fixed cost, but whether the whole task gets cheaper depends on the model. Marking the 46 tools defer_loading and adding openrouter:tool_search lets the model search tool names and descriptions by regular expression and call a match on its next turn, with a pre-search cost of 578 tokens on the Claude models, 144 on GPT-6 Luna and 130 on Gemini 3.8 Flash. Over the whole task, Sonnet 5.5 fell from 59,417 to 30,334 tokens and GPT-6 Luna from 116,282 to 25,544, while Gemini 3.8 Flash rose from 35,409 to 37,568 and Claude Haiku 5.5 rose from 99,319 to 154,623 because in all three runs it asked list_issues for a field the tool does not offer and retried.
Perspective
The findings cover one GitHub task, one GitHub MCP server and four models, Claude Haiku 5.5, Claude Sonnet 5.5, GPT-6 Luna and Gemini 3.8 Flash, with three runs per combination and medians reported; the authors suggest running mcp_cost.py on your own server before deciding.
Tool Search is in beta, conflicts with a forced tool_choice, and is rejected by the Chat Completions API; the 4,000-character cap worked only because the needed fields sat near the top, and combined with Tool Search it left Gemini 3.8 Flash and Haiku 5.5 with one correct answer in three runs each; Haiku 5.5's Tool Search cost ranged from 2,134 to 13,225 tokens across five runs.