GraphCLaRa adds a graph reranker ahead of a frozen CLaRa reader: up to 6.6 F1 gain on multihop QA with fewer documents
Synopsis
Retrieval-augmented multihop question answering suffers from noisy context, and CLaRa's compression of retrieved documents into dense latent memory makes the structural relations between documents harder to exploit. GraphCLaRa places a roughly 1.3M-parameter graph reranker before the frozen CLaRa reader: it builds a query-aware document graph, propagates evidence with a graph attention network, and adaptively decides how many documents to pass on. Across four QA benchmarks it raises answer F1 on three, by 6.6 on 2Wiki, 6.4 on MuSiQue and 2.5 on NQ, while using fewer documents, and trails by 1.1 F1 on the retrieval-saturated HotpotQA.
Interpretation
The retrieval stage of multihop QA gains a front-end module that only selects documents and leaves the generator untouched, so answer quality can change with the reader fully frozen. Previously CLaRa passed all 20 candidate documents to the frozen reader in shuffled original order, with no external reranking and no adaptive budget. On the full evaluation sets of four benchmarks, GraphCLaRa raises F1 from 25.9 to 32.4 on MuSiQue, from 42.7 to 49.4 on 2Wiki and from 56.6 to 59.1 on NQ, with exact match rising in step by 6.8, 5.1 and 2.3; on HotpotQA it falls from 69.5 to 68.5.
The approach organizes candidate documents into a graph that changes with the question, lets documents that are both mutually similar and relevant to the query pass evidence to each other, then ranks and decides how many to read. A graph attention network is a network that aggregates features over graph edges with attention; here it propagates document features along weighted edges to produce a graph feature for each document. The graph is built deterministically from frozen BGE embeddings, with edge weight equal to pairwise cosine similarity times the mean query relevance of the two documents and pairs below a 0.3 threshold dropped; ranking scores come from a gated fusion of a content residual and a graph residual, and the budget head chooses among 5, 10 and 20.
The gains appear on datasets where retrieval still has room to improve and where retrieval failure directly sinks the answer, not on every dataset. HotpotQA's baseline retrieval is already near saturation, with all gold documents inside the top 5 in 77.0% of cases, leaving little to recover. On MuSiQue and 2Wiki the baseline retrieves the full gold set only 15.2% and 39.5% of the time, and GraphCLaRa pushes R@5 to 0.953 and 0.997; on HotpotQA the budget head collapses to 5 documents on 99.3% of instances, and those 5 keep the full gold bridge 74.0% of the time.
Perspective
Evaluation is confined to the four QA datasets released in apple/CLaRa multi stage, with a fixed pool of 20 candidate documents per example, the frozen CLaRa-7B-E2E reader and the frozen BGE-large-en-v1.5 encoder; the authors propose downstream-aware budget objectives, richer budget features and query decomposition that creates retrieval headroom as next steps.
The budget head is supervised by a retrieval proxy, the smallest K covering the gold set, so it is blind to cases where a larger K would answer better; on HotpotQA it collapses to 5 documents, and on the 26.0% of instances that drop at least one gold document F1 falls from 0.646 to 0.479; the only comparison is the full-context CLaRa baseline, with a BL@K control and stronger baselines such as BM25 and cross-encoders left for future work; the path-supervision and edge-refinement terms are not evaluated end to end.