# In-Context Learning

Engineering design often has only 10 to 1,000 evaluated designs because simulations and experiments are costly. GenPFN trains no new generator: a frozen TabPFN classifier reads labeled designs, candidates are interpolated between desired designs and updated with classifier gradients and Gaussian noise, and every candidate is evaluated by the problem's own constraints and objectives. Across fourteen engineering problems, GenPFN v1 has higher class satisfaction and lower realism distance than the tested CTGAN, TVAE and TabDDPM; in the stern case, 993 of 1,000 candidates passed geometric checks and met the propeller-placement area requirement, versus 28 of 2,000 random context designs.

Engineering design often has only 10 to 1,000 evaluated designs because simulations and experiments are costly. GenPFN trains no new generator: a frozen TabPFN classifier reads labeled designs, candidates are interpolated between desired designs and updated with classifier gradients and Gaussian noise, and every candidate is evaluated by the problem's own constraints and objectives. Across fourteen engineering problems, GenPFN v1 has higher class satisfaction and lower realism distance than the tested CTGAN, TVAE and TabDDPM; in the stern case, 993 of 1,000 candidates passed geometric checks and met the propeller-placement area requirement, versus 28 of 2,000 random context designs.

An engineering design study holds only tens to hundreds of simulations, each on its own mesh, and conventional surrogates are retrained for every study. PhysICL is pretrained once on a procedural prior, then treats a few labeled simulations as context and predicts fields, with quantiles, directly at each query's own unstructured-mesh nodes in one forward pass, with no weight update. On five PLAID datasets unseen in pretraining its field error falls as context grows and drops below the nearest context simulation; on AhmedML surfaces its peak GPU memory stays at 4.27 GiB.

An engineering design study holds only tens to hundreds of simulations, each on its own mesh, and conventional surrogates are retrained for every study. PhysICL is pretrained once on a procedural prior, then treats a few labeled simulations as context and predicts fields, with quantiles, directly at each query's own unstructured-mesh nodes in one forward pass, with no weight update. On five PLAID datasets unseen in pretraining its field error falls as context grows and drops below the nearest context simulation; on AhmedML surfaces its peak GPU memory stays at 4.27 GiB.

TabOmics is an open tabular foundation model with 7.3M parameters, pretrained only on synthetic tables, that processes all features retained after preprocessing jointly in a single forward pass instead of subsampling or compressing them. Across 46 classification tasks from three high-dimensional, low-sample-size benchmarks, the eight-member TabOmics v2.5 reaches a mean AUROC of 0.943, above 0.942 for eight-member TabPFN-3.5; the single-member version reaches 0.941 with no significant paired difference, at roughly five times the runtime per forward pass.

TabOmics is an open tabular foundation model with 7.3M parameters, pretrained only on synthetic tables, that processes all features retained after preprocessing jointly in a single forward pass instead of subsampling or compressing them. Across 46 classification tasks from three high-dimensional, low-sample-size benchmarks, the eight-member TabOmics v2.5 reaches a mean AUROC of 0.943, above 0.942 for eight-member TabPFN-3.5; the single-member version reaches 0.941 with no significant paired difference, at roughly five times the runtime per forward pass.

Imbue lets an already-trained Bayesian experimental design policy take in external knowledge available only at deployment without retraining: user-specified priors and observations from previous instances of the experiment are encoded as extra input tokens, processed in-context with the accumulated observations by a shared Transformer, and a reliability filter scores each token against those observations and excludes inconsistent ones. Across four benchmarks it accelerates early-stage information acquisition when the external knowledge is reliable and stays close to the no-external baseline when it is not.

Imbue lets an already-trained Bayesian experimental design policy take in external knowledge available only at deployment without retraining: user-specified priors and observations from previous instances of the experiment are encoded as extra input tokens, processed in-context with the accumulated observations by a shared Transformer, and a reliability filter scores each token against those observations and excludes inconsistent ones. Across four benchmarks it accelerates early-stage information acquisition when the external knowledge is reliable and stays close to the no-external baseline when it is not.

The pretrained time-series foundation model Chronos-2 was tested as a surrogate for power-system dynamic simulation. On 1,800 load-disturbance samples from an IEEE 9-bus system, retrieval-augmented in-context learning beat random reference selection, and fine-tuned Chronos-2 had lower errors than a traditional spatiotemporal graph neural network baseline on the 178 held-out samples, with power-balance regularization improving every reported metric except voltage-angle MAE. Its 35.5 ms inference was 563 times faster than the 20 s ANDES simulation.

The pretrained time-series foundation model Chronos-2 was tested as a surrogate for power-system dynamic simulation. On 1,800 load-disturbance samples from an IEEE 9-bus system, retrieval-augmented in-context learning beat random reference selection, and fine-tuned Chronos-2 had lower errors than a traditional spatiotemporal graph neural network baseline on the 178 held-out samples, with power-balance regularization improving every reported metric except voltage-angle MAE. Its 35.5 ms inference was 563 times faster than the 20 s ANDES simulation.

Combinatorial perturbation screens grow quadratically in pairs, so most combinations are never measured, yet combined transcriptional responses often depart from the sum of the two single effects. GrnPFN is pretrained on simulated screens generated from random gene-regulatory networks to predict that nonadditive residual directly, then incorporates measured pairs through in-context learning without retraining. Across five single-cell screens, with no measured pairs in the target screen it reached a pooled cosine similarity 0.049 higher than gradient boosting trained on other screens (97.5% CI, 0.038–0.060); as measured pairs accumulated, kernel ridge fitted to the same pairs caught up.

Combinatorial perturbation screens grow quadratically in pairs, so most combinations are never measured, yet combined transcriptional responses often depart from the sum of the two single effects. GrnPFN is pretrained on simulated screens generated from random gene-regulatory networks to predict that nonadditive residual directly, then incorporates measured pairs through in-context learning without retraining. Across five single-cell screens, with no measured pairs in the target screen it reached a pooled cosine similarity 0.049 higher than gradient boosting trained on other screens (97.5% CI, 0.038–0.060); as measured pairs accumulated, kernel ridge fitted to the same pairs caught up.

NVIDIA released Kumo Relational, a pre-trained relational foundation model that predicts directly on enterprise relational databases without task-specific training, feature engineering, or flattening tables into one wide row. Across 41 tasks on 15 databases it beats supervised relational deep learning models and expert-engineered gradient-boosted tree pipelines, and on the SAP SALT enterprise benchmark it exceeds AutoGluon ensembles by 8% in mean reciprocal rank. A prediction is stated as a declarative query, and the model answers it in a single forward pass using historical examples as context.

NVIDIA released Kumo Relational, a pre-trained relational foundation model that predicts directly on enterprise relational databases without task-specific training, feature engineering, or flattening tables into one wide row. Across 41 tasks on 15 databases it beats supervised relational deep learning models and expert-engineered gradient-boosted tree pipelines, and on the SAP SALT enterprise benchmark it exceeds AutoGluon ensembles by 8% in mean reciprocal rank. A prediction is stated as a declarative query, and the model answers it in a single forward pass using historical examples as context.