Fairness beyond the final pick: one persuasive argument shifts models' willingness to abandon the right candidate by up to 42 points

Fairness evaluation gains a second observation point: a model can give the same answer in two scenarios that differ only in demographic identity and still switch away from it at different rates under the same argument. Standard demographic fairness benchmarks compare whether a model still selects the same candidate after identities are swapped, treating the final answer as the whole of fairness. Across 15 evaluated models, when the stronger candidate was majority-coded and the weaker minority-coded, a prefix favoring the weaker candidate more readily made the model abandon its correct choice; with identities reversed, the same kind of prefix was less effective, with an average gap of 6.9 percentage points under implied identity and 9.3 points under explicit labels.

FairStress separates a decision into scenario, identity presentation and persuasion, applies each variation, and compares matched pairs. An adversarial prefix is a persuasive statement appended to the prompt that argues for one candidate through authority, social consensus, purported evidence, legal concerns or historical disadvantage. The 300 two-candidate scenarios cover 29 domains including hiring, healthcare, lending, education, housing and criminal justice, and every prefix has a matched version making the same case for the other candidate; identity ranges from none to a demographically coded name, an indirect implication or an explicit label, and the combinations exceed 13 million test cases, with a 50K-item FairStress-Core subset selected using item response theory.

FairStress targets an equality-centered definition of fairness: when the evidence supports a choice, a model should apply the same criteria across identities and resist adversarial prefixes equally. It does not capture every conception of fairness, such as equity-based approaches that treat historical discrimination or underrepresentation as relevant. The benchmark is available for download and built into Olmo-eval so researchers can diagnose asymmetries and test proposed fixes.

A training-data audit found very few examples of decisions based on merit rather than demographics and unequal portrayals of demographic groups, but whether these patterns cause the observed behavioral gaps remains an open question. The activation analysis covers only two models and a handful of identity contrasts, which the authors treat as narrower than the behavioral results. Most debiasing interventions did not reliably reduce the asymmetry, and some traded substantial accuracy for more balanced choices; counterfactual chain-of-thought narrowed the gap in abandoning correct decisions but models still favored minority-coded candidates on ambiguous cases.