Pre-training can act as a grid knob: on 2048 GPUs, curtailment falls from 1.077 to 0.014 MWh

LLM pre-training load can be dispatched by the grid as a flexible load, not just a flat high-power consumer. Most prior data-center load models describe computing demand as continuously adjustable power or generically delay-tolerant jobs, which overstates the flexibility of LLM pre-training. In a 2048-GPU case with two LLaMA pre-training tasks, checkpoint-aware 3D dispatch reduces operating cost by 20.02% and renewable curtailment from 1.077 to 0.014 MWh relative to fixed continuous training.

The scheduler lets training wait or resume only at checkpoints, and the power level after resumption is set by feasible 3D-parallel GPU modes. Checkpoints supply only the safe temporal boundaries for pausing, while recovery capability comes from discrete execution modes combining tensor, pipeline, and data parallelism. After memory, divisibility, and GPU-capacity screening, 135 feasible configurations are generated for LLM-A and 111 for LLM-B; Pareto pruning retains 12 and 21, and the optimal schedule activates only four and three modes.

The framework targets grid-connected, renewable-direct-supply AI data centers, assumes the GPU pool is dedicated to pre-training, and studies a 2048-GPU case with two LLaMA pre-training tasks using CAISO renewable and price profiles. The authors propose extending it to multi-day stochastic dispatch, other AI workloads such as fine-tuning and inference, idle-GPU opportunity cost under shared GPU pools, and coordinated dispatch among multiple AI data centers.

The numerical profiles and generated 3D modes are constructed from public traces and data rather than proprietary production traces; checkpoint value diminishes, with a final cost improvement of only 1.51% and 12 checkpoint segments capturing most available flexibility; a GPU feasibility threshold exists, where 768 to 1152 GPUs cannot complete both daily tranches and 1280 GPUs restore feasibility; and when the renewable-to-AIDC ratio rises to 1.5, green share reaches 89.14% but cost rises slightly and curtailment reaches 2.127 MWh.