Output parity alone misses how predictions are built: adding an attribution-disparity penalty improves all six medical-imaging fairness trade-offs
Fairness in medical image classification is usually enforced only on output-level group gaps, yet with a linear classification head the output score gap equals the signed sum of feature-attribution differences, so opposing contributions can cancel and make a model look fair. Adding the same attribution-disparity penalty to three equalized-odds surrogates produced positive paired hypervolume gains in all six comparisons on HAM10000 gender fairness and Harvard-GF race fairness, with relative gains of 1.49% to 10.57%. Under a predefined utility constraint, the augmented model families also cut equalized-odds difference by 9.0% to 38.7% while attribution disparity and the cancellation ratio fell.