Clinical imaging decisions get a harm gate: HC-ROCP cuts severe-case misrouting from 4.68% to 0.54%
Synopsis
A clinical image classifier that returns a set of possible diagnoses does not guarantee that a severe case avoids a harmful action. HC-ROCP adds a harm penalty weighted by predicted severe-class probability to frozen classifier probabilities, tests coverage and severe-action error on disjoint calibration splits, and then certifies, reports descriptively, or refuses each claim. On COVID-19 radiographs across ten calibration seeds, severe-action errors fall from the exact ROCP baseline's 4.68% to 0.54%, at higher modeled loss.
Interpretation
Clinical imaging decisions now carry a check aimed at severe cases: a policy must not only cover the true diagnosis but also keep the frequency of harmful actions on severe cases within a budget. Earlier risk-optimal conformal prediction chose prediction sets and actions under a robust loss alone; label coverage and average loss do not directly constrain how often a severe case receives a harmful action. In retrospective evaluation on COVID-19 radiographs, severe-action error falls from the exact ROCP baseline's 4.68% to 0.54% across ten calibration seeds, while modeled loss rises from 0.394 to 1.080.
The construction builds a family of candidate policies on frozen classifier outputs, then audits their coverage and harm claims on non-overlapping calibration data. HC-ROCP keeps ROCP's robust action rule and adds a penalty to the set-selection objective that activates only for sets whose robust action lies in the harmful set and is weighted by the classifier's evidence of severity. Calibration data are split by class into construct, coverage, and safety parts; coverage and harm are tested with one-sided Clopper–Pearson tests under multiplicity adjustment, and all three datasets use frozen ConvNeXt V2-Tiny classifiers.
The audit returns one of three outcomes: a supported global claim is certified, an unsupported one falls back to a descriptive policy, and an unsupported subgroup claim is refused. At a 5% harm budget, global claims are certified for COVID-19 and ISIC, while ISIC's rare squamous-cell-carcinoma subgroup claim is refused; at 1%, all three settings receive only descriptive reports. ISIC's squamous-cell-carcinoma safety set has only 42 cases, and even with zero errors its adjusted upper bound stays above the budget; the MEL subgroup has 302 safety cases and a much narrower bound.
Perspective
The evaluation is limited to three retrospective imaging settings: COVID-19 radiographs, ISIC 2019 dermoscopy, and EyePACS-to-APTOS retinopathy transfer; actions and loss matrices were prespecified by five clinical experts, and results are compared against observed disease labels rather than real patient outcomes.
Calibration seeds share the same classifier and test observations; validity requires within-class i.i.d. sampling and the gate-cutoff conditions, and targets the realized calibration mixtures, so changes in prevalence or workflow require reassessment. The objective has no cardinality penalty, so large prediction sets can limit diagnostic specificity, exact enumeration scales exponentially with label count, and robustness across classifier backbones remains to be evaluated.