Knowledge-graph prompts don't need the whole graph: relation knowledge picks a dozen facts, and seven open LLMs answer better
Synopsis
Knowledge-graph question answering usually ranks facts by relevance and packs them into the prompt, but relevance guarantees neither sufficiency nor consistency. This method uses relation knowledge, either hand-written rules for what relations mean and how they compose or a mapping learned from training questions annotated with relation paths, to search the graph for a compact fact set that derives an answer, then applies a derivability check for whether an answer follows and a consistency check for whether facts conflict, with a validation-chosen gate deciding when the derived answer replaces the LLM's. On WebQSP, seven open LLMs read these contexts, 14.7 facts on average, at 10.2 exact-match points above SubgraphRAG's 100 facts.
Interpretation
Which facts go into a knowledge-graph prompt now has a compact version selected by relation knowledge: it keeps only the facts on a path that derives the answer, rather than truncating a relevance ranking. Earlier retrievers kept the most relevant facts, path generators kept the paths a fine-tuned model planned, and explorers let the LLM judge whether its facts sufficed; relevance ranking alone establishes neither that the facts suffice to derive an answer nor that they do not conflict. On WebQSP these contexts hold 14.7 facts on average, and seven open instruction-tuned LLMs read them at 10.2 exact-match points above SubgraphRAG's 100 facts on average, 4.6 to 13.0 points per LLM; on CWQ, 18.6-fact contexts are read at about the same exact match as SubgraphRAG's 100 facts while using a quarter of the prompt tokens.
Beyond selection sit two checks outside the LLM: a derivability check asks whether an answer follows from the context, a consistency check asks whether the context's facts conflict, and together they decide whether the derived answer or the LLM answers. Relation knowledge has two sources: CLUTRR ships kinship rules, while the other benchmarks have none and learn a mapping from training questions and their annotated relation paths, a word-to-relation lexicon with a lift threshold for templated questions and a vote of the most similar training questions' chains for human-written ones. On held-out WebQSP and CWQ questions, the derived answer is right for 81.5 and 62.1 percent of questions where the derivability check passes; also requiring exactly one derived answer raises precision to 89.8 and 95.9 percent on 17.6 and 4.9 percent of questions, confirmed on questions never used for tuning under a pre-specified plan.
Perspective
The demonstrated scope is knowledge-graph question answering on five public benchmarks: CLUTRR kinship stories, PathQuestion and MetaQA templated questions, and the human-written WebQSP and CWQ, read under fixed prompts by seven 4-bit-quantised open instruction models plus GPT-5 and several reasoning models; WebQSP and CWQ use RoG's released question subgraphs with topic entities given.
The derived answer's reach on human-written questions is small and depends on seen chains: 94.7 percent of WebQSP test questions have a relation chain seen in training, and of the 63 with only unseen chains the check passes 4, all wrong; on CWQ the check passes 11.8 percent, and no comparative or superlative question passes with a right answer. The consistency check can judge a fact by path evidence only where a second path joins its two entities, true of 2.5 percent of named-entity fact mentions in the compact WebQSP and CWQ contexts; if the graph itself holds a wrong fact and the selector picks it, both checks almost always miss it.