Perovskite model scoring gets a new split: a lookup table matches the published best on random splits, and neural models overtake trees on unseen institutions

Machine-learning evaluation for perovskite solar cells now scores models on splits grouped by device provenance: duplicates kept together, then whole publications, then whole institutions, plus a temporal split that trains on earlier publication years. Earlier models on this archive each selected their own records, were evaluated mainly with random splits, and were typically compared only against other neural models, so reported accuracies could not be compared directly and each carried an unknown contribution from recalled duplicates. Replicating the published state-of-the-art evaluation setup, a lookup table that does no learning and predicts each device from the median of its duplicates in training reaches a mean absolute error of 2.934%, against the reported state-of-the-art value of 2.936%.

On the institution-grouped split, SciBERT+attention, which reads the semantic meaning of layer names with a scientific language model, reaches the lowest error at 2.912%, ahead of CatBoost with native categorical handling and the gradient-boosted baseline. On the looser random, duplicate-grouped and publication-grouped splits, tree ensembles lead because they treat layer names as opaque categorical strings, which is efficient for recognizing familiar recipes. On the temporal split, where test devices average 13.59% efficiency against 10.54% in training, every neural model improves on the gradient-boosted baseline by 0.19% to 0.27% and is more accurate than every tree-based model.

The predictive information comes from the absorber composition, the device architecture and the fabrication process as three groups of inputs, not from absorber chemistry alone. On the random split, removing the device stack raises the error by 0.342%, far above the 0.092% for the fabrication process and 0.060% for the composition, and most of that inflated value comes from layer identity. On the institution-grouped split, removing the fabrication process, the device stack and the absorber composition raises the error by 0.091%, 0.082% and 0.074% respectively, putting the three contributions close together.

The protocol addresses researchers and engineers who train and compare models on the public perovskite device archive, covering 23,620 curated devices, the corresponding-author institution for 94.7% of 4,683 publications, and five splits: random, duplicate-grouped, publication-grouped, institution-grouped and temporal. The authors recommend that claims of generalization or physical discovery be supported by the institution-grouped or temporal split, that tree ensembles serve as standard baselines, and that comparisons be controlled against a lookup-table baseline to quantify remaining memorization.

The authors note that tree-based models encode complex material compositions merely as statistical variables and lack a physical understanding of chemistry, so the finding that device architecture and fabrication methodology matter as much as absorber chemistry is a structural limitation of the baseline model, and modeling the absorber physics requires further study with composition-aware neural architectures. On the temporal split about half of the later devices carry an input absent from training, creating a level offset in the baseline's predictions that makes permutation-importance magnitudes not directly comparable with removal magnitudes.