Correct answers aren't enough: AA-PersonalityBench scores how models write by default, ranking 8 indexes from 1 to 100

Model differences are no longer judged only by whether an answer is right, but also by how a model writes by default. AA-PersonalityBench isolates personality and communication style so two equally correct answers can be compared on style. Prior evaluation centered on answer correctness, yet two models can give equally correct answers and still feel very different to use. The benchmark sets 30 questions for each of 30 traits, 900 in total, and every model answers all 900 once with no system prompt, so default behavior is what gets measured.

Scores come from blind pairwise comparisons rather than grading single responses directly. Judges see the trait, its definition, the question and two anonymous answers, but no model names and no reasoning text. A panel of LLM judges supplies the verdicts, with one judge per comparison returning A, B or tie, and the order of the two responses is randomized to avoid position bias. Comparisons per trait are fitted into Elo scores and reported as percentile ranks with 95% confidence intervals; when the bounds overlap, the models are not definitively different.

The 30 traits roll up into 8 personality indexes, and 21 communication characteristics are counted programmatically from the response text. Indexes average trait z-scores, with the sycophantic trait inverted in the Transparency index; communication characteristics cover length, structure, punctuation and emphasis such as bold, italics, emoji, acronyms and closing questions. These are measured across all 900 responses, for example average bold text per response ranges from 8.7 to 17.2 among the listed models.

The benchmark measures how a model writes by default with no system prompt; scores describe how strongly a trait shows, not whether it is desirable, since traits can be positive, neutral or negative depending on subjective preferences. Questions are sent as single user messages, each model answers all 900 once, and judges see only outputs, not reasoning text. The full question set is private to avoid contamination, with two sample questions published per trait.

The benchmark states that when confidence interval bounds overlap, models are not definitively different, so some ranking gaps are not firm distinctions. Trait scores depend on subjective preference, and the benchmark measures strength of expression rather than desirability. The question set stays private to avoid contamination, with only two sample questions per trait published.