Zero-shot defect detection drops the sliding window: RADIOlogy scores an image in one forward pass and leads training-free baselines on industrial and medical benchmarks
Synopsis
Zero-shot anomaly detection has leaned on multi-scale sliding-window inference over CLIP, or on prompt learning that needs segmentation masks from auxiliary datasets. RADIOlogy instead exploits the patch–text alignment that multi-head attention pooling encoders preserve out of the box, scoring the average similarity gap between normal and abnormal prompts so that a single forward pass yields both an image-level anomaly score and a pixel-level anomaly map. Across seven industrial and seven medical benchmarks it reaches mean image AUROC of 92.6 industrial and 97.3 medical, with pixel AUROC of 95.2 and 87.3.
Interpretation
Zero-shot anomaly detection can run without training and without windowing: one forward pass per image produces both an image-level anomaly score and a pixel-level anomaly map. The prior training-free line, anchored by WinCLIP, ran multi-scale sliding windows and extracted the [CLS] token over many local crops per image to work around weak patch-level vision-language alignment in standard CLIP, at the cost of inference latency and compute. Across seven industrial and seven medical benchmarks, RADIOlogy averages image AUROC of 92.6 industrial and 97.3 medical, and pixel AUROC of 95.2 and 87.3; in the same table WinCLIP averages image AUROC of 79.9 industrial and 85.5 medical, and pixel AUROC of 77.9 and 69.2.
Scoring compares the gap between average similarity to a set of normal prompts and to a set of abnormal prompts, rather than an absolute similarity, and that gap decides whether a region is anomalous. The prompt ensemble is inherited from WinCLIP, so the change comes from the encoder and the scoring rule rather than from redesigned prompts. The backbone is C-RADIOv4-h, each image is encoded once at 448 resolution, and pixel maps are evaluated after bilinear upsampling and Gaussian blur, with one set of numeric hyperparameters shared by all fourteen benchmarks.
Pixel localization stacks three signals from the same forward pass: the patch-text margin, diffusion over a content-adaptive affinity graph built from the trunk's dense features, and spherical k-means part clustering on SAM3-aligned features. Square-window pooling mixes objects with background and yields low-resolution defect boundaries; weighting neighbors by semantic relatedness lets a defect spread to its true extent. On the industrial benchmarks, adding context pooling, affinity refinement and region pooling in turn raises pixel AUROC from 89.5 to 93.7, 94.6 and 95.2, and max-F1 from 24.3 to 28.9, 33.6 and 37.8.
Perspective
The demonstrated setting is training-free zero-shot: no target-domain training samples are available, and evaluation covers seven industrial and seven medical benchmarks, with HeadCT, BrainMRI and Br35H used for medical detection and ISIC, ClinicDB, ColonDB and TN3K for localization. The foreground prior assumes a nominal background along the image border.
The foreground prior fails for off-center objects that touch the image border, and the authors list depth estimation heads as a future direction. A performance gap persists between detection and localized segmentation, with multi-resolution token representations and fine-tuning the C-RADIOv4 backbone proposed as next steps. The pipeline is also sensitive to extreme aspect ratios, so KSDD inputs are split into square bands along the long axis and scored separately.
