Meta's new ads retrieval engine scores over a billion ads per second on one GH200 server, lifting Facebook conversions 2.4%
Synopsis
Meta has released the next generation of Andromeda, its personalized ads retrieval system, designed jointly across model, system and GPU hardware. Retrieval previously could not afford ranking-stage model depth at its scale; the new architecture brings large-language-model-style mixture-of-experts into retrieval, approaching ranking-level performance at roughly 300x fewer inference FLOPs than the 2024 retrieval model. The combined single-GPU pipeline processes over one billion ads per second on one server, which Meta calls a new throughput benchmark for retrieval-stage models, and Meta reports a 2.4% conversion increase on Facebook since 2025.
Interpretation
The first stage of the ads pipeline can now score every eligible ad with near ranking-stage model quality while still narrowing the pool in milliseconds, rather than filtering on coarse statistics alone. Ranking-stage deep modeling had been too costly to apply at retrieval scale; the 2024 Andromeda grew model capacity by four orders of magnitude but did not bring ranking-level accuracy to retrieval. Meta reports the new architecture approaches ranking-level performance in a typical retrieval-then-ranking system at roughly 300x fewer inference-time FLOPs than the 2024 retrieval model.
The throughput gain comes from fusing model computation and system logic into one GPU kernel and co-locating index matching, scoring and top-K selection on a single GPU server. Previously each user-ad computation was split across many small kernels, with intermediate tensors generating terabytes per second of high-bandwidth memory traffic, and ad tensors shipped between local CPU hosts and remote accelerators saturating the network link. The combined single-GPU pipeline processes over one billion ads per second on a single GPU server, cuts network traffic by roughly 100x and returns the system to a GPU-compute-bound regime.
Joint optimization with NVIDIA pushed pipeline components toward the current hardware ceiling, with the gating network and index matching each gaining several times in throughput. Before optimization the gating kernel ran at 2.4 billion ads per second, and data movement between index matching and scoring accounted for a significant fraction of end-to-end latency. The optimized gating kernel rose from 2.4 billion to over 10 billion ads per second, reaching about 85% of peak HBM bandwidth; index matching improved over 2x to over 100 billion ads per second; INT8 quantization added up to 2x throughput over optimized FP16 kernels with accuracy on par with 16-bit precision.
Perspective
The report covers throughput and conversion figures from Meta's production ads retrieval system and its joint optimization of pipeline components with NVIDIA on the Hopper platform, for readers interested in co-design of recommendation systems and GPU systems. Meta says it is extending the optimization work to Grace Blackwell and evaluating how sensitive end-to-end performance is to CPU-to-GPU compute ratios.
Meta says the pipeline is not yet at its ceiling, and the benchmark of multiple times current production throughput still depends on further optimizations reaching production; the INT8 gain is stated as up to 2x with accuracy only reported as on par with 16-bit precision. Migration to Grace Blackwell is still under evaluation, and the effect of CPU-to-GPU compute ratios has no conclusion yet.