Where semantic priors should enter causal discovery: orientation gains hold, but query placement and admission certificates do not yet
Synopsis
Observational data often leave causal directions unresolved or unstable, and large language models can propose directions from variable names and descriptions. This study splits the use of semantic information into query allocation, proposal admission and graph retention, and tests each on the Asia, Sachs, Child, Insurance and Alarm benchmark networks. Semantic orientation beats coin-flip orientation, but targeting unstable edges shows no established advantage over matched-random allocation; the tested admission certificate can only abstain given the available calibration tasks, and learned references contain the true DAG in only three of 80 planned tasks.
Interpretation
Semantic information helps settle causal directions, but where to ask for it is a separate question that is not yet settled. Under an unrestricted linear-Gaussian model, observational data cannot distinguish opposite orientations within an equivalence class, so semantic input enters as supplementary information, and semantic proposals themselves can be wrong. Across five benchmark networks, semantic orientation improves Directed-F1 over coin-flip orientation by +0.08263, while coin-flip over the base is also +0.10363, so comparing against the base alone would overstate what semantics supplies.
Selecting query targets by finite-sample instability shows no stable advantage over randomly chosen committed edges under the same response budget. All four primary estimator and sample-size settings have equal-network 95% intervals that include zero, and the point estimates change sign. The F1+F2 versus F1+rand contrast is compared in the PC-500, PC-2000, BOSS-500 and BOSS-2000 settings, and the equal-network estimate across the five primary networks is +0.00474 with an interval that includes zero.
Semantic orientation does improve recovery on the edges already selected, and that gain is distinct from the quality of query placement. Replacing the directional evidence on the same F2 targets with semantic majority votes instead of bootstrap counts gives positive point estimates in all four primary settings. Only BOSS-500 has an interval above zero among the four primary settings; on the Barley, Hailfinder and Win95pts families, semantic resolution raises Directed-F1 over bootstrap-only resolution by 0.418 percentage points.
Admission risk and downstream retention need separate calibration, and whether retention keeps the true graph depends on whether the reference already contains it. The tested finite-grid Hoeffding certificate cannot support any nontrivial admission on the real calibration pools, which hold ten F1 and nine F2 graph tasks. In the learned-reference study, only three of 80 planned tasks have a stored reference containing the true DAG, and exact membership among the 72 downstream-usable tasks is 0.0428.
Perspective
The conclusions rest on the five primary benchmark networks Asia, Sachs, Child, Insurance and Alarm plus the three additional families Barley, Hailfinder and Win95pts, using unrestricted linear-Gaussian structural equations, the PC and BOSS estimators, and sample sizes of 500 and 2000; the reported 95% network intervals are exploratory and not adjusted for multiple comparisons.
The authors note the small number of benchmark families, possible semantic familiarity with the benchmarks, and incomplete primary protocol records; the admission-certificate failure concerns the tested certificate and available independent graph tasks rather than a universal sample-complexity barrier; the retention guarantee is marginal, and pruning cannot restore a true graph absent from the stored reference.