Creativity benchmarks are really testing the tail: scoring only a benchmark's rarest items tracks human preference better
Sorting creativity benchmarks by how rare their items are under a reference corpus makes the tail subset's model ranking agree more closely with LM Arena creative-writing human preferences than the peak subset, so creativity evaluation can be read as task performance in the long tail.