# Autoregressive Pretraining

Visual generation is split into two stages: autoregressive pretraining acquires visual knowledge, and diffusion post-training renders it into pixels. In controlled comparisons at roughly 400M parameters, the autoregressive-then-diffusion route reached 65.5 on GenEval, above 61.2 for pixel diffusion trained from scratch, and a semantic tokenizer needed neither an image decoder nor a reconstruction loss to support rendering. Allocating compute by fitted scaling laws, a roughly 3B-parameter text-to-image model trained in about 6,600 H100 GPU-hours scored 0.86 on GenEval and 87.78 on DPG-Bench.

Visual generation is split into two stages: autoregressive pretraining acquires visual knowledge, and diffusion post-training renders it into pixels. In controlled comparisons at roughly 400M parameters, the autoregressive-then-diffusion route reached 65.5 on GenEval, above 61.2 for pixel diffusion trained from scratch, and a semantic tokenizer needed neither an image decoder nor a reconstruction loss to support rendering. Allocating compute by fitted scaling laws, a roughly 3B-parameter text-to-image model trained in about 6,600 H100 GPU-hours scored 0.86 on GenEval and 87.78 on DPG-Bench.

Predicting cancer treatment side effects has long relied on static nomograms built from a handful of variables. SPARC instead models longitudinal sequences of blood counts, metabolic panels and vital signs, pretraining on 101,920,379 measurements from 224,047 patients and then fine-tuning for mortality, anemia, neutropenia, febrile neutropenia, acute kidney injury and liver toxicity. It matched or led comparison models on held-out, prospective and multi-hospital cohorts, and its predicted risk strata identified patients most likely to benefit from G-CSF prophylaxis.

Predicting cancer treatment side effects has long relied on static nomograms built from a handful of variables. SPARC instead models longitudinal sequences of blood counts, metabolic panels and vital signs, pretraining on 101,920,379 measurements from 224,047 patients and then fine-tuning for mortality, anemia, neutropenia, febrile neutropenia, acute kidney injury and liver toxicity. It matched or led comparison models on held-out, prospective and multi-hospital cohorts, and its predicted risk strata identified patients most likely to benefit from G-CSF prophylaxis.