Rerun the same semantic query and the answer can change: up to 24% of rows flip at temperature 0, and plans amplify that into most answers

Running the same semantic query again can return a different answer even with decoding set to temperature 0. Semantic query engines treated one execution as a property of the system, assuming a rerun returns the same answer because benchmarks execute at temperature 0. In SemBench's five published executions, 21% of text results report a different quality in at least one execution, including engines with deterministic plans; on gpt-oss-120B, Movie Q3 returned 4 different counts over 10 executions and its quality metric ranged from 0.64 to 0.93.

Whether a change reaches the answer depends on the plan, not on the model alone. Benchmarks reported one execution or the mean of several, without per-item outputs or a test of whether their comparisons are stable. On UDA-Bench, Llama-70B (M) changes 4.0% of extracted values yet the result table of 83 of 123 queries differs between executions; gpt-oss-120B changes 7.5% of values and 68% of answers while quality moves by a mean range of 0.031.

Changes concentrate on items close to a tie, and one execution's decision margin or reasoning length flags them. Output differences at temperature 0 were treated as uniform noise or as state that persists through an execution. The 533 duplicated rows sent as identical requests within one execution disagree in 6.0% of gpt-oss-120B executions against 6.1% across executions; Granite-8B's decision margin predicts later changes with AUC 0.93 and gpt-oss-120B's reasoning length with AUC 0.84.

Repetition reduces the variance but cannot remove it, and recording operator outputs is what makes an answer reproducible. Voting or repeated executions were treated as a way to remove randomness, and semantic outputs were cached for reuse. Averaged over queries, a vote of three removes 36% of the pairwise disagreement and a vote of five 48%, yet no two combined executions return the same answer; re-asking the longest-reasoning 20% of items gets 64% of a full vote's reduction at 1.4 calls per item.

The measurements cover semantic queries over text and tabular data on three benchmarks, SemBench, LRO-Bench, and UDA-Bench, executed through their own harnesses without changing plans, prompts, or scoring. Per-item logs require a hook into an engine's model client, which the authors built for LOTUS and LRO-Bench's client; other engines enter only through SemBench's published results. For readers who need stable answers, the reproducible path is to record the operator outputs that produced an answer, at about 39 bytes per output.

The authors could not run Gemini 2.5 Flash, so the per-item analysis rests on six open-weight deployments and two commercial models; no reachable model runs with and without reasoning at temperature 0, so reasoning cannot be separated from the model. The deployment effect rests on one model served by two deployments. Five executions bound the smallest detectable flip rate and give answer reproduction a granularity of 0.1; flip rates keep growing, from 3.1% at five executions to 5.2% at forty for Granite-8B on five Movie queries. The cascade replay fixes the proxy's scores to one execution and so understates the variance of a real cascade.