Skip to main content
Search

# Benchmark redundancy

1 related articles

Subscribe to updates
  1. aclanthology.org

    Dropping a "redundant" benchmark can delete the wrong one: two tests correlated at 0.86 barely agree on top models

    Pruning a benchmark because it correlates with another may discard exactly the signal that separates frontier models. Across 4,497 models on Open LLM Leaderboard v2, BBH and GPQA correlate at 0.861 overall, yet conditional Spearman falls to 0.434 among the top 20% and to -0.278 among the top 5%; 7 of 15 pairs are best fit by zero-tail-dependence Frank copulas, so high bulk correlation does not imply agreement at the frontier.

You’re all caught up