# F1 Ranking Certification

After labeling only the items where two classifiers disagree, two agreement counts yield exact bounds on the F1 difference, enough to certify which classifier to pick within a stated tolerance or to say which agreement labels still need querying. Across 3,200 frozen model pairs, bounds that share one label completion certified 79.46% of choices in the reserved cells at a 0.01 tolerance, versus 6.57% for separate bounds; under magnitude ordering, the 0.01 tolerance cut total labels for all pairs from 121.16 to 74.59, against 500 for full labeling. The guarantee covers the evaluated batch, not future data.

After labeling only the items where two classifiers disagree, two agreement counts yield exact bounds on the F1 difference, enough to certify which classifier to pick within a stated tolerance or to say which agreement labels still need querying. Across 3,200 frozen model pairs, bounds that share one label completion certified 79.46% of choices in the reserved cells at a 0.01 tolerance, versus 6.57% for separate bounds; under magnitude ordering, the 0.01 tolerance cut total labels for all pairs from 121.16 to 74.59, against 500 for full labeling. The guarantee covers the evaluated batch, not future data.