Skip to main content
Search

# Physical-Engineering's Last Exam

2 related articles

Subscribe to updates
  1. Mercor

    AI agents meet real engineering drawings: the best configuration reaches 48.3% on a hidden test set

    Mercor and Google's AI & Infrastructure research team released Physical-Engineering's Last Exam (PLE), 85 long-horizon tasks testing whether AI agents can do mechanical, electrical, civil and structural engineering work. Each task runs in a containerized project folder of about 61 files, including PDFs, spreadsheets, emails, 2D CAD drawings, 3D BIM models and planted outdated revisions, with no internet access and up to six hours. The best configuration, Claude Opus 5.5 in Claude Code, reaches 48.3% on the hidden test set at an average of 27 million tokens and 63 minutes per task; its designs met 91% of individual constraints on average, but only 41% cleared every hard check.

  2. Agents take on real physical-engineering projects: all 14 frontier models fall below 50% on PLE's private test set

    Google AI & Infrastructure, with Mercor, released Physical-Engineering's Last Exam (PLE), an 85-task benchmark and open competition spanning mechanical, electrical, civil and structural engineering that tests whether agents can complete real engineering work under a licensed engineer's review. Each task is a containerized project folder averaging 61 files, and an agent must find governing codes, run calculations and produce deliverables, earning points only after passing every hard safety check. On the private test set all 14 frontier models scored below 50%, with the top configuration reaching a mean of 48.3% on its native harness.

You’re all caught up