Pothole Detectors Score 0.619 on the Published Split and 0.467 Without Near-Duplicates: A Video-Disjoint Re-Split of BharatPotHole

The published split of the public pothole benchmark BharatPotHole keeps test images closely related to training images, so detection scores partly reflect recognition of seen scenes. The dataset had been reported for detector performance only, without the scale of duplication and train–test overlap or a comparison of detection performance under alternative partitioning protocols. An audit of all 7,074 released images found 86 exact duplicates and 2,769 export copies of other frames, 40.4% in total; only 1,333 of 4,196 positive images were unique frames; in the published split 38.8% of test images had a near-duplicate in training and 99.4% came from training videos.

The same detector scores markedly lower on the test subset without near-duplicates, so part of the published-split score comes from leakage. The model, weights and training data are held fixed and only the test images change, so the difference is linked to near-duplicates and shared source frames. For YOLO26n under the published protocol, AP50 on positive images was 0.619 on the published test set, 0.739 on the leaked subset and 0.467 on the near-duplicate-free subset; the paired difference between the published test set and the near-duplicate-free subset was 0.152 (95% CI 0.090–0.234), a relative gap of 33%, and the gap ranged from 0.152 to 0.174 across five definitions of that subset.

A controlled contamination test shows directly that copies of training frames can multiply detection scores on an unseen test set. The video-disjoint models had seen no C-test image or video, each substituted image kept its own exported annotation, and the number of negative images stayed constant. At a replacement share of 100%, AP50 rose from 0.133 to 0.494±0.024 for YOLO26n and from 0.135 to 0.436±0.030 for YOLO26s; with copies of unseen C-test frames as the matched control, the two models reached 0.149 and 0.150.

The higher internal score under the published split did not carry over to better detection on other roads. The published-protocol model was trained on more images yet scored lower on external positive images. On the 1,530 positive images of the RDD2022 India subset, the published-split model reached AP50 0.072 (95% CI 0.062–0.085), against 0.099 (0.087–0.113) and 0.097 (0.086–0.110) for the video-disjoint models, with non-overlapping intervals.

The findings rest on one dataset, BharatPotHole, and on two detectors of one family, YOLO26n and YOLO26s, each trained with one seed; the leakage mechanism does not depend on the detector, but the size of the gaps will vary with dataset, architecture and training procedure. The same-model comparison was made for YOLO26n alone, and no training-size-matched comparison of Protocol A and Protocol C was made.

The gap between the published test set and the near-duplicate-free subset is reported as an observed gap, not as the total causal effect of leakage, because the subsets differ in composition and shared videos affect both. C-test holds 134 positive images from nine videos, one of which contributes 57, so its confidence intervals are wide. RDD-India contains only positive images and differs from BharatPotHole in image size, blur and object scale, so its results reflect domain shift as well as training differences. Annotation inconsistencies were not corrected and lower all absolute values reported.