Deep Learning Foundation Models for Low-Data Regimes from Classical Molecular Descriptors

It proposes supervised pretraining on low-noise, calculable molecular descriptors as a route to rich, highly transferable molecular representations. Rather than relying on other pretraining signals or objectives, this work anchors the pretraining signal directly on classical molecular descriptors, offering a new entry point for molecular foundation models. The strategy is realized concretely as the CheMeleon model and evaluated on 58 benchmark data sets, making it a methodological contribution with an explicit implementation and multi-data-set validation.

CheMeleon enables directed message-passing neural networks to finally exceed the performance of classical methods in the low-data regime. Deep learning had previously been unable to outperform classical machine learning methods on practical, real-world benchmarks with limited training data; this work reports a shift in that performance pattern. The conclusion rests on rigorous statistical comparisons across 58 data sets against classical baselines such as Random Forest on molecular fingerprints and descriptors.

CheMeleon also outperforms existing foundation models in statistical comparisons. Beyond comparison with classical baselines, the evaluation includes existing foundation models, indicating the gains from this pretraining strategy are not only relative to traditional methods. Evaluation spans 58 data sets with rigorous statistical comparisons, and the model is described as O(10M) parameters.

The CheMeleon model and the pretraining framework are open-sourced to encourage adoption and extension of this pretraining strategy across chemical sciences. Releasing both the model and the framework makes the strategy reproducible, transferable to other chemical tasks, and open to further extension. As a public release of research outputs, it provides a directly usable basis for subsequent adoption and extension.

The work targets prediction of properties relevant to small-molecule drug discovery, with evaluation on 58 data sets sourced from the industry-led Polaris benchmarking initiative, and it applies to low-data regime settings where training data are limited; its value lies in providing a reusable model and framework for adopting and extending this pretraining strategy across chemical sciences, especially for research and development settings that require molecular property modeling under data scarcity.

The available text is summary-level and does not include specific data set names, metric values, statistical test details, or model architecture and pretraining configurations; readers would therefore still watch how the strategy performs across different property types and data scales, and under what conditions it extends to other chemical domains, which would need to be confirmed against the methods and results of the original paper.