PCOS screening stops hiding behind one accuracy number: rarest subtype recall climbs from 75% to 87.5%

A screening model for polycystic ovary syndrome cannot be judged by one overall accuracy number, because the same model behaves very differently across clinical subtypes. Prior work in this area almost always answered only whether a patient has the condition, reporting accuracies between 80% and nearly 99%, and rarely checked whether that accuracy held across the four Rotterdam phenotypes. On a public dataset of 541 patients with 177 positive cases, the tuned random forest's per-phenotype accuracy ranged from 75.0% for the rarest phenotype to 97.8% for the most common, a 22.8-point spread that was 23.3 points after removing the seven phenotype-defining variables, so the gap is not an artifact of how the labels were built.

Switching the prediction target from whether a patient has the condition to which subtype she belongs does not solve the problem; it moves the errors onto a different group of patients. A class-weighted six-class random forest approached perfect accuracy on common phenotypes, but recall on the rarest phenotype fell from 75.0% under the binary model to 43.8%, and unclassified cases fell from 38.9% to 31.5%. The six-class model reached a macro-F1 of 0.661 and 83.9% overall accuracy, with balanced accuracy of 63.2%; nine of its sixteen rarest-phenotype patients were classified directly as controls, as were 37 of 54 unclassified patients.

Letting the binary task and the subtype task share one hidden representation recovers rare-subtype recall without giving up overall accuracy. The multi-class model trained alone performed worst on the rarest phenotype, while the shared representation lets the larger binary task, trained on all 541 patients, support the data-poor subtype task. Under the same ten-fold out-of-fold predictions, the model held 90.2% binary accuracy and 0.962 AUC, raised rarest-phenotype recall to 87.5% and unclassified recall to 79.6%, with paired tests giving p=0.016 and p<0.0001 for those two gains and no significant difference for the common phenotypes.

An ablation attributes the improvement to feature selection and class weighting rather than to the shared hidden layer itself. Representation sharing had been treated as the reason multi-task training helps, but removing the shared layer left rare-phenotype recall higher, not lower. The phenotype-only variant reached 71.7% balanced accuracy, 0.659 macro-F1, 87.5% rarest-phenotype recall and 77.8% unclassified recall; the no-shared-layer variant reached 89.8% binary accuracy, 0.956 AUC, 93.8% rarest-phenotype recall and 81.5% unclassified recall, and across five seeds its rarest-phenotype recall was 92.5% (std 4.7) against 90.0% (std 5.0) for the shared version.

This is a methods study on a retrospective public benchmark, and readers should treat it as a reminder about evaluation rather than a ready screening tool. All results come from one public dataset of 541 patients from a single Indian state, with subtype labels derived by proxy rules from existing features rather than biochemically confirmed diagnoses. The authors state that none of the numbers constitute evidence of clinical benefit in a real diagnostic pathway, and they list the next steps: external validation on a cohort with biochemically confirmed phenotypes, a paired statistical comparison against current binary screening practice, and a decision-analytic evaluation.

The dataset lacks biochemical androgen measurements such as total testosterone, free testosterone and DHEAS, so phenotype labels rely on a proxy built from hirsutism, acne and relative hormone levels; the rarest phenotype has only sixteen patients, giving wide confidence intervals; and nearly one-third of positive patients fit no phenotype, which the authors treat as a dataset artifact rather than epidemiological reality. The authors also note that the hyperparameter search budget was limited and that the shared hidden layer is not necessary for the rare-phenotype improvement.