Self-consistency backfires on long context: seven of eight samples agree on the same wrong answer
On long-context tasks, self-consistency, the practice of sampling several answers and aggregating them, no longer improves performance and often lowers it, because positional bias makes the samples repeat the same mistake and the vote turns that mistake into a consensus.