Biomedical knowledge-graph QA stops relying on document retrieval alone: three systems run in parallel and merge, lifting satisfaction from 36% to 86%

With the same models, running document, graph and agentic retrieval in parallel and merging their answers answers more questions that require the underlying experimental data than document retrieval alone. The earlier GNQA indexed only unstructured text describing GeneNetwork and never gave the model direct access to the structured experimental data, so questions needing the dataset itself rather than papers about it stayed out of reach. On 40 expert-curated questions with reference answers, with full use of Claude Opus 4.8, plain document retrieval reached a satisfaction frequency of 36% against 86% for the combined system.

The system rewrites GeneNetwork's metadata from a relational database into an RDF graph, so the model can write SPARQL queries to fetch structured data instead of only reading pre-converted text. The graph and agentic graph retrieval are cognitive architectures, meaning the model plans and acts on the structured data; the agentic version inspects returned results and revises its queries over several steps before answering. Evaluation ran on 40 questions over three repeats, with Claude Haiku 4.5 as a single LLM judge scoring precision, recall and f1, which are combined into the satisfaction frequency ψ.

Delegating most inference to a local model and calling a frontier model only for hard steps approaches the frontier-only result at markedly lower cost. With the local model Qwen3 30B A3B alone, graph retrieval reached a satisfaction frequency of 17% and agentic retrieval 16%. Pairing Qwen3 30B A3B with Claude Opus 4.8 raised graph retrieval to 43% and agentic retrieval to 53%, and the combined system reached 59%, against 86% for the combined system running Claude Opus 4.8 throughout.

The system targets biomedical and systems-genetics researchers who need to query GeneNetwork data, answering questions about model organisms, datasets and mapping results through a chat interface. The benchmark consists of 40 questions and reference answers of varying difficulty, curated by domain experts from GeneNetwork's resource classification scheme. The authors propose developing the combined system further so it selects subsystems by task rather than running all three in parallel, and widening the set of models it can route between.

ψ is a fixed-threshold summary of the LLM judge's precision, recall and f1, and the authors state it was not calibrated against actual user satisfaction judgments; the 70% and 60% thresholds and the judge's scoring were not validated against human raters. The benchmark was drawn from the same resource classification scheme used while developing the RDF graph and the four systems, with no independent held-out test set, and the authors say they cannot rule out bias. Statistical comparisons use three repeats, giving only two degrees of freedom, and the combined system's advantage on Claude Haiku 4.5 and GPT-OSS 20B is reported as inconclusive rather than as evidence of no difference.