Math reasoning traces aren't decoration: suppress the graph's reasoning nodes and answer entropy rises across 15 open-weight models

Intermediate expressions inside a math reasoning trace are treated as testable causal mediators rather than as one block of text. Earlier perturbations of reasoning traces were mostly random, such as replacing numbers or altering demonstrations, and did not distinguish which expressions actually feed the derivation. Across GSM8K, MATH500 and AIME, suppressing all reasoning nodes significantly increased answer entropy, with p-values below 10 to the minus 12.

The method pulls the mathematical expressions out of a trace and links them into a directed acyclic graph, where nodes are expressions and edges say which one is computed from which. This causal CoT graph is extracted automatically with the symbolic parser SymPy, then expanded backward from the answer to the question by matching expressions, and finally the edges are reversed. The dataset holds 1671 problems, 983 from GSM8K, 384 from MATH500 and 304 from AIME, averaging 9 to 40 intermediate reasoning nodes and 6 to 10 hops from question to answer per problem.

Models appear to favor the paths this graph identifies, rather than merely absorbing the prompt. The study compares the probability of graph-aligned reasoning paths against equal-length random paths, instead of looking only at final accuracy. On MATH500 and other splits, most models show a spike at the 100th percentile, meaning a substantial fraction of reasoning paths consist entirely of higher-probability transitions.

The work targets the analysis of mathematical reasoning traces, covering 1671 problems from GSM8K, MATH500 and AIME, with 15 open-weight models from 1B to 70B parameters on the model side. The extraction pipeline relies on symbolic parsing, and geometry or diagram-based problems were filtered out. The dataset and supporting artifacts are released on the project page for follow-up graph-aligned intervention experiments.

The authors note that about 10% of samples degenerate after pruning into a trivial graph containing only the answer node, requiring manual intervention; such edits touched 88 of roughly 40K parsed vertices and 71 of roughly 300K extracted edges. The method does not directly apply to geometry, abstract mathematics or commonsense reasoning. Model behavior also splits into exponential and bell-shaped regimes, the latter tied to a minority of low-probability transitions, which the authors partly attribute to intrinsic uncertainty in the reasoning process while not ruling out extrinsic factors such as poor modeling or hallucination.