Skip to main content
Search

# Long-Horizon Agent Benchmarks

2 related articles

Subscribe to updates
  1. arXiv

    Remory adds residual memory tokens to summaries, lifting SummHay citation F1 by 4.04 points and approaching full-context joint score

    Remory trains a neural memory network to append a bounded sequence of soft memory tokens after a generated summary, helping a frozen LLM approximate the continuation it would produce with the full history; on SummHay it raises citation F1 by 4.04 points and joint score by 3.55 using about 5.2% of input positions, and yields consistent gains for Qwen3.8-27B and GLM-5.3-Flash on long-horizon agent benchmarks while reducing repeated tool outputs and tool errors.

  2. arXiv

    Argo-Bench tests data agents on a simulated 235-table, 7.5-billion-row delivery warehouse, and the best model solves only 34.8% of tasks

    The authors built Argo-Bench: a 2024 New York City food-delivery platform simulated into an Oracle E-Business Suite-style warehouse of 235 tables and 7.49 billion rows, where agents are graded on the actions they file — banning accounts, allocating incentive budgets, filing forecasts — against the simulator's withheld ground truth; among 14 frontier and open-weight models, the strongest, Claude Opus 5.5, solves 34.8% of tasks and averages 59.5 points.

You’re all caught up