Agent workflows stop being rebuilt per query: a pool plus a contextual bandit picks one, cutting MATH training tokens at least 40.55%
Synopsis
Workflow selection for LLM agents is recast as a contextual bandit over a reusable pool, reaching an 81.37% average across five benchmarks while cutting MATH training-phase token overhead by at least 40.55%.
Interpretation
An LLM agent no longer generates a workflow from scratch for every query; it picks one from a pool built in advance. Earlier automatic optimization either fixed one workflow per task type or treated each query as a fresh generation problem. Across MATH, GSM8K, MBPP, HumanEval and GAIA, the average score is 81.37%, above MaAS at 79.79% and AFlow at 78.76%.
The pool comes from execution-guided evolutionary search, and the selector conditions on embeddings of the query and the workflow code. Selection is modeled as a contextual bandit scored by a hybrid UCB that adds a linear exploration bonus to a neural reward prediction. Pool coverage is 72.3% on MATH, 98.0% on GSM8K, 93.9% on HumanEval and 95.3% on MBPP.
The training-phase savings come from sample efficiency rather than from shortening inference. On MATH, training-phase token overhead drops by at least 40.55%, and end-to-end training overhead including one-time pool construction drops by at least 29.73%. On MATH, an exploration coefficient of 800 gives 54.36% solve rate at about 6.14M tokens, while 400 gives 52.43% at about 4.28M.
Perspective
This report covers the full text, including the method, experiments on five benchmarks, ablations, cross-domain transfer and backbone scaling. It is written for researchers and engineers building LLM agent workflows who care about selection policy and training cost. The workflow pool is built from the training set by execution-guided evolutionary search, and pool coverage is reported per dataset.
The hybrid UCB uses the original embedding space for its exploration bonus, and the authors state that this cannot directly inherit neural-bandit regret bounds, so the bonus is not a calibrated confidence interval. Pool coverage on MATH is 72.3%, lower than on the other benchmarks, which the authors attribute to the difficulty of competition-level mathematics. The optional online learning mode adds latency and cost, and the authors recommend triggering it judiciously.