Skip to main content
Search

# Hardware selection for training jobs

1 related articles

Subscribe to updates
  1. arXiv

    Meta's LEAP predicts training throughput at submission: 77% median accuracy over 30 nightly retrains, replacing a lookup table

    Meta's LEAP predicts a training job's throughput on each candidate hardware type at submission time and ranks the candidates by throughput per unit cost for the cluster scheduler. It replaces a lookup table keyed on model type, trainer count, batch size and hardware type, which returned an estimate for only 13% of job attempts. Over 30 nightly retrains the median accuracy is 77%, and on the 946 jobs the old table covered, LEAP predicts 85.1% within ±20% of measured throughput against 53.2% for the table.

You’re all caught up