Hamburg University of Technology

The study evaluates nine LLM variants across three families (GPT-4.1, GPT-5, Gemini 2.5) on three MoleculeNet datasets (Delaney solubility, Lipophilicity, QM7 atomization energy) using a systematic blinding procedure that iteratively reduces available information, complemented by 0-, 60-, and 1000-shot in-context sample sizes, in order to determine whether molecular property prediction reflects genuine in-context regression or verbatim retrieval of memorized target values, and it adds positive and negative controls for the memorization experiments, structural reference baselines for the multi-shot experiments, and bootstrap confidence intervals for all results; it finds no evidence of verbatim retrieval on these legacy benchmarks and shows that blinding exposes conflicts between pre-traine

The study evaluates nine LLM variants across three families (GPT-4.1, GPT-5, Gemini 2.5) on three MoleculeNet datasets (Delaney solubility, Lipophilicity, QM7 atomization energy) using a systematic blinding procedure that iteratively reduces available information, complemented by 0-, 60-, and 1000-shot in-context sample sizes, in order to determine whether molecular property prediction reflects genuine in-context regression or verbatim retrieval of memorized target values, and it adds positive and negative controls for the memorization experiments, structural reference baselines for the multi-shot experiments, and bootstrap confidence intervals for all results; it finds no evidence of verbatim retrieval on these legacy benchmarks and shows that blinding exposes conflicts between pre-traine