Agents stop guessing tool calls: compiling the catalog into a planner lifts four benchmarks by up to 28.9 points
Synopsis
LLM agents usually pick tool calls by trial and error, with no guarantee that a plan can execute. CGAP compiles the tool catalog deterministically into an abstract PDDL model that tracks only which fields are available, lets a classical planner order the calls, and has an LLM add entity-specific calls whose arguments and results are checked. Across ToolSandbox, ComplexFuncBench, BFCL multi-turn and τ-bench retail with three LLMs, it beats a standard tool-calling agent, by up to 28.9 points.
Interpretation
For a single request, the order of tool calls is now decided by a classical planner rather than chosen step by step by an LLM. ReAct-style agents and plan-first systems such as HuggingGPT, ReWOO and LLMCompiler use the LLM for both request interpretation and planning, so nothing guarantees a call's inputs are available when it runs. Across four benchmarks and three LLMs, CGAP beats the plain tool-calling agent in all 15 model-metric comparisons with paired intervals excluding zero; on ToolSandbox the gain is 13.9 to 28.9 points.
The planning model records only whether a field has a source, not its value, so call order can be fixed before any tool output is known. The catalog is compiled deterministically into PDDL actions, with required inputs as preconditions and candidate outputs as effects; an LLM turns the request into one goal per requested result, and breadth-first search returns a shortest abstract plan. In the weather example, three forecasts, two date resolutions and two city lookups need seven calls, while the abstract plan has five actions because one city lookup makes coordinates available for every forecast.
During execution an LLM adds the entity-specific calls the abstract plan omits, and a runtime checks preconditions, argument schemas and sources before each call and verifies each result afterward. Every argument must cite a conversation span or a field of an earlier successful response and is resolved by a fixed decoder or an application-supplied deterministic transformation; the LLM never writes an argument value. The runtime rules guarantee that every admitted execution maps onto a valid abstract plan, and a NoPlan result certifies that no call sequence passing the checks completes the request for the compiled model.
Perspective
The demonstrated setting is stratified subsets of four interactive tool-use benchmarks plus all 114 τ-bench retail tasks, with three LLMs and majority-of-three aggregation; state-dependent planning is realized only on ToolSandbox using hand-written device-dependency annotations.
The guarantees hold only for the compiled model and runtime rules, and NoPlan certifies infeasibility only for the catalog as compiled; request parsing can misread an operation, a cited source can be wrong or tied to the wrong entity, and the verifier can accept an incorrect result and cannot undo a side effect already made; how often turns end in NoPlan rather than Limit is unmeasured.