Meta's LEAP predicts training throughput at submission: 77% median accuracy over 30 nightly retrains, replacing a lookup table

In a mixed-vendor accelerator fleet a training job can often run on several hardware types, and the best choice depends on its throughput on each; LEAP produces that estimate before the job runs and ranks candidates by predicted throughput divided by computational cost. The prior method was a rule-based lookup table keyed on model type, trainer count, batch size and hardware type; it held entries for only 19 model types and produced an estimate for 13% of job attempts on a held-out day. On a held-out subset of one day with 7,131 jobs from 276 model types, the table covered 946 jobs; on those 946, LEAP predicted 85.1% within ±20% of measured throughput against 53.2% for the table, and within ±10% the figures were 72.1% and 33.4%.

The predictor uses only information available at submission: the job's launch configuration, the specifications of each candidate hardware type, and the throughput the same job measured in earlier runs. Throughput spans seven orders of magnitude and a single regressor fits the extremes poorly, so the model bins log throughput and fits a set of gradient-boosted trees to separate throughput ranges. The model splits log throughput into 13 equal-frequency bins, iteratively fits a median regressor per bin while reassigning rows, then trains a classifier to place a job in a bin from features alone; it is retrained nightly on about 750,000 job attempts from the last 30 days and scored on the following day, with a median accuracy of 77.0% over 30 retrains and individual nights between 73.5% and 80.6%.

The demonstrated scope is one production training fleet: about 25,000 job attempts a day across more than 660 model types and more than 20 hardware types, with accuracy weighted by job attempts, so it reflects the many small jobs more than the few large jobs that account for most GPU-hours.

A new hardware type cannot be ranked reliably until it has run deployed jobs, and holding out one hardware type costs 37 points of accuracy; accuracy is also low on model families absent from training, dropping 49.7 points on the one family where the cascade was refit. The authors also note that accuracy varies by about seven points from night to night, so a single-day comparison can reverse the next day.