Enterprise agents stop re-reading documents per question: a knowledge tree plus ontology lifts Harvey LAB rubric pass rate from 55.1% to 84.2%

Enterprise agents answering complex questions no longer start from scratch on retrieved passages; they first locate information on a pre-compiled structured knowledge layer. The prior setup was retrieval-augmented generation: documents are split into chunks and indexed, the most relevant chunks are passed to the model at question time, passages lose their surrounding context, facts across files are not joined, and the work repeats for every query. On Harvey's LAB benchmark with GLM 5.3 as the agent model, answers met 84.2% of expert rubric criteria against 55.1% for agentic RAG, and average task cost fell from $0.64 to $0.28, above Harvey's own harness reported at 79.1% for GPT-5.6 Sol and 71.7% for Opus 4.8.

The structure has two layers: one lets an agent navigate the corpus like a filesystem, the other records domain entities and relationships in a queryable form. The knowledge tree turns each long document into a folder with an index listing title, summary and page ranges per section; the domain ontology defines entity types, properties and relationships, such as a transaction carrying its kind, status, value and underwriting basis. On four public benchmarks with GLM 5.2 over the same parsed documents, knowledge-tree navigation versus flat hybrid retrieval scored 75.5% to 91.3% on BrowseComp-Plus (830 questions), 84.3% to 90.4% on BRIGHT-Pro (739 questions), 82.1% to 88.7% on FinanceBench (150 questions) and 33.8% to 51.6% on OfficeQA Pro (133 questions).

The ontology layer moves filtering and joining into structured queries, so accuracy rises while tokens and tool calls fall. With the knowledge tree alone the agent explored through repeated reading and reasoning, using more than twice the tokens of the RAG baseline; with the ontology, filtering happens in the database. Across the three LAB configurations, the ontology setup used 14.2 tool calls, 0.86M tokens, 8.6 minutes and $0.28 per task, the knowledge tree 59.2, 3.79M, 30.8 minutes and $1.18, and agentic RAG 36.5, 1.80M, 30.0 minutes and $0.64; the ontology setup answered all 250 tasks while agentic RAG left 9 unanswered.

The demonstrated scope is knowledge question answering over enterprise documents and structured data: LAB's simulated law firm with 9,288 files, 266 matters, 46 clients and 250 requests, tau-knowledge's simulated bank with 698 documents and 97 all-or-nothing tasks, and four public retrieval benchmarks. The fabric can run inside an organization's own environment, including small air-gapped setups using smaller models; the ontology is drafted by agents and reviewed by domain experts, with every change versioned and the domain's evaluation set rerun before an update goes live.

The knowledge tree did not solve completeness; the agent still has to work out which documents describe the same thing and whether it has found every relevant record, which is what the second layer addresses. On tau-knowledge the fabric completed 39 tasks against 38 for the baseline, matching accuracy rather than improving it. The authors say they are still improving compilation accuracy and efficiency and name shared persistent memory as the next step.