Finance agents meet an open world: none of 150 sessions passes, and all 35 the simulated user approved fail a program check
FinCosmos opens all three uncertainties of financial agent evaluation at once: a simulated investor reveals a hidden strategy only when asked, the agent must source its own Web data and backtest under a data cutoff, and programs verify the delivery instead of matching a reference answer. Across 150 sessions with four coding agents and a tool-free model, none is accepted, and all 35 sessions the simulated user signed off fail at least one program veto.