One Space Decides a Multiple-Choice Score: Correct-Answer Likelihood Falls by Orders of Magnitude, Accuracy Usually Moves Less Than Half a Point
In multiple-choice evaluation, whether the space after the prompt's colon belongs to the prompt or to the option changes the probability the model assigns to the correct answer. Keeping the space with the option matches the tokenization seen in pre-training; keeping it with the prompt lowers the correct answer's log-probability by several orders of magnitude. Accuracy changes by less than half a point in six of nine model-benchmark pairs, but Qwen3.5 loses almost ten points on PIQA.