Splitting document QA into a searcher and a reader: same model gains 11.0 F1 on M3DocVQA
Synopsis
Open-domain multimodal document QA requires first locating the right passages and then reading them, yet current retrieval-augmented agents give one model both kinds of tools and let it schedule itself. KnowForge separates the pipeline by job: a search-only Locator agent, a code-scheduled bridge judge that forces the missing second hop, a bounded evidence pool merging every search ranking, and a Deep Reader that only reads text, images and tables. With a self-hosted Qwen3.6-35B-A3B, it reaches F1 47.5 on the full M3DocVQA (2,441 questions, 3,368 PDFs), 11.0 above the published M3DocRAG and 7.1 above one-shot retrieve-and-answer on the same index and model.
Interpretation
Document QA is split into specialized locating and reading agents, each restricted to one kind of tool, instead of one model both searching and reading on its own schedule. Earlier retrieval-augmented agents handed both tool types to a single model and let it schedule itself. On the full M3DocVQA (2,441 questions, 3,368 PDFs), a self-hosted Qwen3.6-35B-A3B (3B active) reaches F1 47.5, 11.0 above the published M3DocRAG system and 7.1 above one-shot retrieve-and-answer on the same index and model.
The pipeline combines a hybrid-retrieval knowledge-point index, a search-only Locator agent, a code-scheduled bridge judge, a size-bounded evidence pool merging every search ranking, a Deep Reader limited to text, images and tables, and a synthesizer with a format checker. The bridge judge is scheduled by code to force the missing second retrieval hop rather than leaving that decision to the model. On MultiHop-RAG it reaches 73.5 accuracy, above the strongest published RAG and GraphRAG results at 71.2; on the single-document MMLongBench-Doc it lifts the same model from 34.8 to 47.0, level with GPT-4o reading page images at 46.3.
The gain from splitting by job can be attributed part by part, and the Deep Reader's value tracks how visual each corpus is. Merging both agents into one with all tools and twice the rounds loses 1.9 F1 because it keeps searching instead of reading. Across the three full benchmarks the Deep Reader is worth 3.8, 9.8 and 0 points; a round-budget sweep shows a step at four Locator rounds and a plateau beyond; query latency scales with model calls rather than corpus size.
Perspective
This report draws on summary-level material and does not cover the complete paper. The demonstrated scope is three benchmarks: full M3DocVQA, MultiHop-RAG and the single-document MMLongBench-Doc, with a self-hosted Qwen3.6-35B-A3B and a separate 53.5 result on DeepSeek-V4-Flash. The three indexed benchmarks are served as read-only knowledge bases with free registration, and prompts and runner code are released.
The Deep Reader contributes 3.8, 9.8 and 0 points across the three corpora, indicating its benefit depends on how visual the corpus is. The round-budget sweep shows a plateau after four Locator rounds, with further rounds adding no gain. Query latency scales with model calls rather than corpus size, a relationship reported at the summary level.