Agentic framework ANA searches autism fMRI pipelines: leave-one-center AUC 0.814 across 17 ABIDE I sites, above ASD-DiagNet and ASD-SAENet
Synopsis
Autism resting-state fMRI classifiers rarely generalize across acquisition sites, and many reported accuracies are inflated by data leakage. ASD Neuro Agentic (ANA) has language-model agents propose features from the literature and researcher input while fixed tools perform every calculation; across 1,028 participants from 17 ABIDE I centers it searched pipelines and selected a ComBat-harmonized ensemble of tangent-space connectivity models. In leave-one-center-out testing the ensemble reached an AUC of 0.814 (95% CI, 0.773–0.841), above ASD-DiagNet and ASD-SAENet.
Interpretation
Pipeline search for autism fMRI classification can now run automatically without leaking held-out labels, and it yields a classifier that works across sites. Earlier ABIDE accuracies were often inflated because feature selection or tuning used held-out participants, and many functional-connectivity differences did not replicate across sites. In leave-one-center-out validation over 1,028 participants at 17 ABIDE I centers, the selected ComBat ensemble reached an AUC of 0.814 with a center-level bootstrap interval of 0.773–0.841; none of 10,000 label permutations reached that performance.
Specialized agents propose candidates while deterministic code computes every number. The language models plan, retrieve literature and propose features and model settings, but take no part in calculating any metric. The summary and the results section give different counts of tested pipelines, 71 and 73 respectively; balanced accuracy rose from about 0.58 to about 0.726 with multi-atlas tangent-space features and 0.741 with the final pipeline, and independent runs reproduced the selected result to four decimal places.
The final model is a greedy ensemble of tangent-space connectivity models from six parcellations, harmonized with ComBat. A single parcellation has limited predictive power, and more complex graph and convolutional networks did not do better on the same cross-validation folds. The ensemble improved AUC by 0.032 over the best single atlas without harmonization (CC400, AUC 0.782; 95% CI for the difference, 0.013–0.044), while GraphSAGE, GCN, GAT, GIN, a CNN and an MLP had AUCs 0.14–0.25 lower than the selected model.
Perspective
The results come from leave-one-center-out evaluation on 1,028 participants (500 with autism, 528 controls) at 17 ABIDE I centers, and the regional analyses used only the C-PAC filt_global preprocessing pipeline. Six centers with 231 participants were evaluated once after the first-phase pipeline was frozen, giving an AUC of 0.768; the final ensemble's 0.801 on those six centers is not an independent test because those centers took part in its development.
The authors note that the analyses used ABIDE I only and the regional analyses used one preprocessing pipeline, so external validation on ABIDE II is needed; the best center-specific settings were selected from several candidates and should be read as exploratory estimates rather than guaranteed accuracy at new sites; ComBat harmonization used unlabeled scans from each held-out center, so the model needs a batch of scans from a new center before it can be applied; and preprocessing included global signal regression, so connectivity findings are relative to the removed global signal. An independent test of the final model on new centers is still needed.