# Failure mode analysis

FrontierPhysics replaces fixed physics questions and paper reproduction with given solutions by 56 end-to-end tasks that 42 physicists built from their own research, spanning nine areas of theoretical, computational and experimental physics. An agent receives only a high-level prompt like a principal investigator's brief, then must review the literature, set a plan, run the research, and deliver both deterministically checkable numerical outputs and a manuscript. Across 22 models in 26 configurations, no configuration passes more than 46 of its 168 rollouts, and manual review of trajectories traces the main weaknesses to insufficiently rigorous reasoning and limited expert intuition rather than gaps in physics knowledge.

FrontierPhysics replaces fixed physics questions and paper reproduction with given solutions by 56 end-to-end tasks that 42 physicists built from their own research, spanning nine areas of theoretical, computational and experimental physics. An agent receives only a high-level prompt like a principal investigator's brief, then must review the literature, set a plan, run the research, and deliver both deterministically checkable numerical outputs and a manuscript. Across 22 models in 26 configurations, no configuration passes more than 46 of its 168 rollouts, and manual review of trajectories traces the main weaknesses to insufficiently rigorous reasoning and limited expert intuition rather than gaps in physics knowledge.