Explanations learn the curve, not just a mask: Ra-NEM tops faithfulness at 0.494 and runs fastest

The quality of an attribution map can now be trained directly against insertion and deletion curves instead of being generated by a heuristic and scored afterward. Earlier attribution methods did not optimize the insertion or deletion curve directly; faithfulness was only a metric applied after the fact. On 1,000 images sampled from the ImageNet validation set, Ra-NEM reached an average faithfulness of 0.494, above RISE at 0.422, NEMt at 0.423, and IBA at 0.455.

The construction rewrites the insertion curve as a top-k feature selection problem, then makes that selection differentiable so gradients can pass through it. Optimizing the area under the curve is non-differentiable, and the usual Gumbel softmax draws only one element at a time, which becomes costly for large k. During training, k is drawn at random per input and Gumbel noise is sampled, so the Gumbel top-k trick returns an ordered set of the top k features in one pass, which is then plugged into the neural explanation mask framework.

Wiring this objective into the neural explanation mask framework yields Ra-NEM, whose masking network uses the same U-Net decoder blocks as NEMt. Earlier NEM variants could only say whether a feature was important, not how important it was relative to others. Evaluation covers four ImageNet-pretrained models, ResNet50, ConvNeXt, VGG16, and ViT, with faithfulness measured on 1,000 images and robustness and randomization checks on a 100-image subset.

The objective can next be attached to attribution algorithms beyond the neural explanation mask framework, since the loss term itself is independent of the specific attributor. Settings that need real-time or high-volume explanations, such as online video analysis, can benefit directly from its low latency and resource efficiency. Binary masking opens a path to text and graph modalities where partial occlusion is ill-defined, though that direction has not yet been tested experimentally.

Faithfulness scores are sensitive to the perturbation scheme, and the same method scores differently under noise, zero, and blur perturbations, even though the ranking appears stable. Ra-NEM trails the top method RISE by 0.139 on VGG16, so it is not best on every model. Training takes longer upfront than NEMt, and larger sample counts k raise faithfulness while slowing training, a trade-off that has to be weighed per setting. The method produces an ordinal ranking of features, so the absolute attribution values do not correspond to the actual change in classifier output.