An open post-training recipe makes fact-checking models harsher on low-status posters, while seven deployed models show no such tilt
Synopsis
On fact-checking claims with randomized poster profiles, the open Tülu-3 post-training recipe makes OLMo-2 32B and Tülu-3 70B judge claims from lower-status posters false more often, an effect under one percentage point but statistically reliable, exceeding training drift and replicating on independent claim sets; Meta's, Alibaba's and Mistral's recipes do not produce it. In OLMo-2 32B the model does not notice the job more but interprets it differently, with a lower-status job pushing the base model toward true and the post-trained model toward false, a flip carried by a single residual-stream direction at supervised fine-tuning. None of seven deployed commercial models doubts lower-status posters more.
Interpretation
Whether post-training makes a model shift its truth judgement with the poster's job depends on the specific recipe, not on post-training as such. It was unclear which post-training changes how much a model's truth judgement depends on the poster's social class, and how that change happens inside the model. Across base and post-trained checkpoints of six model families on fact-checked claims with randomized poster profiles, only the open Tülu-3 recipe (OLMo-2 32B, Tülu-3 70B) shows the doubt; Meta's, Alibaba's and Mistral's recipes do not.
The effect is under one percentage point but statistically reliable. That size has to be separated from drift caused by training itself. The effect exceeds training drift and replicates on independent claim sets.
In OLMo-2 32B the change comes from how the job is interpreted, not from paying more attention to it. A lower-status job pushes the base model toward true and the post-trained model toward false. The flip is carried by a single residual-stream direction, happens at supervised fine-tuning, and was confirmed on 400 new claims with pre-specified predictions.
Perspective
The findings rest on fact-checked claims with randomized poster profiles, cover base and post-trained checkpoints of six model families, and concentrate the mechanism analysis on OLMo-2 32B.
The effect is under one percentage point, statistically reliable but small; the mechanism confirmation was done on OLMo-2 32B, and whether other model families share the same residual-stream direction is not stated in the material.