LiveMACEBench evaluates five frontier LLM agents over 30 days of live markets and finds return rankings diverge sharply from capability metrics
The work introduces LiveMACEBench, a process-aware benchmark that uses live cryptocurrency and U.S. equity markets as a naturally evolving testbed in which five frontier LLMs run continuous paper-trading trajectories under matched Base, Tool Use, Persistent Memory, Rule Following, and Multi-Agent Collaboration configurations, evaluated through both realized outcomes and trace-derived diagnostics; across roughly 30 days it finds a pronounced outcome–capability gap, with return rankings fluctuating and often diverging from capability-specific measurements while similar outcomes arise from markedly different patterns of mechanism use.