Black-box LLM backdoors now show up in reasoning graphs: GCoTD lifts TPR by 3.44 points on GPT-4o

Backdoor detection no longer looks only at a model's final output; it looks at the dependency structure formed when the model decomposes its reasoning into steps. Earlier backdoor detectors mostly relied on internal activations, trigger inversion or output-level signatures, requiring white-box access or prior knowledge of the attack mechanism and fitting poorly to closed-source models reachable only through an API. Across four attack types and six models, including LLaMA-3, DeepSeek-V3, Qwen-3 and GPT-class models, GCoTD reaches the highest or near-highest TPR in most settings, with TPR gains of 3.44 and 3.75 percentage points over the prior state of the art on GPT-4o and LLaMA-3-8B respectively.

The pipeline first uses a few-shot template to steer the target model into rewriting its response as a graph, where each reasoning step is a node and dependencies between steps are directed edges. The detector is a lightweight language model with MiniLM-L6-v2 as encoder and a binary classification head, fine-tuned with LoRA on labeled clean and backdoored reasoning graphs. Training data spans six attack methods under four attack types, DPA, WPA, HSA and CoTA, with 500 poisoned and 500 clean samples per attack, giving 6,000 labeled reasoning graphs.

The detector judges graph-level anomalies: unusually dense clusters, logic discontinuities along edges, and reasoning trajectories that contradict the initial query's constraints. An ablation shows graph-structured input yields an 11.5 percentage point TPR gain over linear chain-of-thought input, so the improvement cannot be attributed to prompting alone. That ablation runs on MMLU with LLaMA-3-8B under BadChain attack: graph input gives 82.3% TPR and 26.6% FPR, versus 70.8% TPR and 33.1% FPR for linear chain-of-thought input.

The demonstrated setting is a query-only target model: the defender has no access to parameters, training data or hidden states, while the detector itself is trained on labeled clean and backdoored reasoning graphs. Evaluation covers 11 benchmark datasets, 6 target models and 6 attack methods, including held-out attacks not seen in training.

Held-out TPR stays above 72% but drops 7 to 9 percentage points relative to in-distribution performance. FPR is higher on the MATH benchmark, which the authors attribute to the inherent difficulty of mathematical reasoning and the diversity of valid solution paths. Template quality and domain relevance affect performance, and there is no fallback when the target model refuses to answer, gives very short responses or fails to produce a sufficiently structured reasoning trace; robustness to truncated or incomplete traces has not been systematically evaluated.