ProtoGestalt ties rare-disease face screening to eyebrows, eyes, nose and cheeks, reaching 0.85 accuracy on matched controls
Synopsis
Rare genetic disorders are often missed at first presentation, and most facial diagnostic models are end-to-end black boxes that cannot say what drove a call. ProtoGestalt adds facial-landmark constraints to PIP-Net prototype learning, so each prototype draws evidence only from one of ten anatomical regions and predictions are sums of those local prototypes. On controls matched by age, sex and ethnicity it separated syndromic from unaffected faces at 0.85 accuracy, versus 0.73 for the region-masked variant; AUPRC fell as controls increased but stayed above 0.70.
Interpretation
Facial-image screening for rare genetic disorders gains an interpretable option: instead of only a syndrome label, the model breaks its call into locatable facial-region evidence. Earlier facial rare-disease diagnosis models DeepGestalt and GestaltMatcher are end-to-end and offer no mechanism for ascribing specific, clinically interpretable phenotypic features to an individual patient or disease. On UTKFace controls matched by age, sex and ethnicity, the global-pooling version separated syndromic from control faces at 0.85 accuracy and the region-masked version at 0.73; on unmatched UTKFace controls the two reached 0.94 and 0.86.
The method adds a hard spatial constraint to prototype learning, so each prototype is responsible for one anatomical region of a face. PIP-Net pools prototype activations with global max pooling, so a prototype contributes wherever its strongest activation falls, and that location can shift between images. Prototypes are assigned round-robin to ten regions, left and right eye, left and right brow, nose, left and right cheek, mouth, chin and forehead, defined by MediaPipe Face Mesh landmark indices; pooling takes the maximum only inside the mask, and the mask has no learnable parameters.
The regional constraint concentrates the evidence but costs recall, a trade-off visible across several backbones. Across five backbones, region masking raised Top-1 accuracy for four of them, while its effect on higher-rank accuracy varied. On IResNet100, region masking gave the highest Top-1 accuracy at 33.56%, while global max pooling gave the highest Top-5 and Top-10 accuracies at 52.27% and 59.86%.
Perspective
Evaluation covers 882 GMDB-Frequent test images and 1309 GMDB-Rare images, with controls from CASIA-WebFace and UTKFace plus a UTKFace cohort matched by age, sex and ethnicity; prototype consistency and clinical relevance were examined only in a Cornelia de Lange syndrome case study.
Clinical status is unavailable for individuals in CASIA-WebFace and UTKFace, who are assumed free of rare disorders and may include undiagnosed cases; a logistic regression on region-masked prototype representations distinguished the control source datasets with AUPRC 0.8, suggesting unmatched controls may inflate apparent screening performance; region masking reduced recall or overall predictive performance in several experiments; prototype consistency has not been evaluated systematically across syndromes.