Parsing without per-page rechecking: a distilled single model passes its teacher at 87.9 on ParseBench
Synopsis
Aleph Alpha distilled its self-checking agentic parsing harness into a single vision-language model, Parsy, which handles each page in one pass. The harness reached 86.0 on ParseBench by running several models and up to three passes per page, too slow and costly for millions of pages a month. Parsy was trained on harness-labeled public documents whose labels were cleaned with an independent layout model, an ink-coverage check and a second OCR model; it scores 87.9 on ParseBench, parses hundreds of thousands of pages per day on one H100, and costs one to two orders of magnitude less per page.
Interpretation
Document parsing moves from several models rechecking every page to one model in a single pass. The harness sent every page through a judge at least once and difficult pages through the loop several times, using several models and up to three passes per page, which meant more GPUs for customers processing millions of pages a month. On ParseBench, 2,078 human-verified pages from real enterprise documents, the harness scored 86.0 in internal evaluation; Parsy scores 87.9 across all five categories with the public scoring code, above the reported 86.6 for LlamaParse Agentic Plus.
Training labels come from the harness's own output, but they are checked by tools that do not share its blind spots. A model trained on parser output inherits that parser's systematic errors, and a judge sharing a blind spot will not catch them. After each round an independent layout model proposes regions, an ink-coverage check finds printed text outside every labeled box, and a second OCR model rereads the same regions; disagreements are flagged, people sample each detector to confirm what it catches, and the corpus is fixed before retraining.
One model in one pass also changes where it sits on the leaderboard. Almost every top ParseBench system is a pipeline running several models and several passes per page, and the next-best vision-language model scores 80.5. Parsy's 87.9 is seven points above that 80.5; the authors also observed that passing Parsy's output through the harness's judge-and-refine loop lowers its ParseBench score.
Perspective
The demonstrated setting is parsing of industrial and public-sector documents with nested tables, footnotes, scanned pages and forms, under a self-hosted deployment requirement; the ParseBench score is produced with the public scoring code across all five categories, and the authors state it would place first on the public leaderboard, though Parsy is currently available only through customer engagements and is not listed there.
The authors attribute the model overtaking the harness to training data that already filtered out the harness's run-time errors, and they observed that passing Parsy's output through the judge-and-refine loop lowers its ParseBench score, so that loop no longer adds quality here; throughput varies with page density, resolution and layout requirements, and no single fixed figure is given.