# AutoSPECTRA

Scientific foundation models usually ship with aggregate benchmark scores, while the performance limits that matter in use surface only after deployment through scattered reports. AutoSPECTRA constrains a general-purpose coding agent into a reproducible protocol: define a measurable generalization axis, run controlled comparisons against fixed reference methods, and confirm on data not used to select the axis. Across four biological foundation models, ESMFold2, STATE, CONCH and AlphaMissense, the audits tied performance shifts to protein sequence composition and relatedness to pretraining data, representation of perturbations in training, effective spatial resolution, and residue-level genomic, evolutionary and structural context, and rejected several initially plausible hypotheses.

Scientific foundation models usually ship with aggregate benchmark scores, while the performance limits that matter in use surface only after deployment through scattered reports. AutoSPECTRA constrains a general-purpose coding agent into a reproducible protocol: define a measurable generalization axis, run controlled comparisons against fixed reference methods, and confirm on data not used to select the axis. Across four biological foundation models, ESMFold2, STATE, CONCH and AlphaMissense, the audits tied performance shifts to protein sequence composition and relatedness to pretraining data, representation of perturbations in training, effective spatial resolution, and residue-level genomic, evolutionary and structural context, and rejected several initially plausible hypotheses.