TensorRT 11.1 adds a Global Performance Tuner: instead of one fixed build route, it searches per-model engines that meet accuracy
Synopsis
NVIDIA released the Global Performance Tuner in TensorRT 11.1, turning compilation scheduling from a single default strategy into customizable build routes. Optimization had relied on heuristics distilled from internal KPI models, and accumulating rules could regress on unseen external models. The tuner validates each candidate solution against a specified accuracy criterion and measures end-to-end inference performance, then selects the fastest solution among those that meet the requirement.
Interpretation
TensorRT performance optimization expands from one shared default scheduling strategy to multiple build routes customizable per model and deployment environment. Engineers previously studied internal KPI workloads, encoded optimizable patterns as heuristics in the default strategy, and new models inherited that shared logic. The same model can compile into different solutions under different scheduling, and different models need not share an optimal route: the material gives two models, one whose best result comes from the third route and another whose best comes from the first.
The tuner treats accuracy as a constraint and end-to-end runtime as the objective, choosing among an enlarged candidate solution set. Performance and numerical acceptability were previously hard to optimize separately, since aggressive scheduling could exceed an application's tolerance while higher-precision choices sacrificed performance. For each candidate solution, the tuner validates outputs against the given accuracy criterion and measures end-to-end inference performance, selecting the minimum-runtime solution only among those that meet the accuracy requirement.
Perspective
The described content concerns the Global Performance Tuner released in TensorRT 11.1 and addresses scheduling and build-route selection in a deep learning compiler; the material motivates it with a mathematical framework and a seasoning analogy, and reports no measured performance or accuracy results for specific models.
The material treats runtime as uniquely determined by a solution under a fixed execution environment, yet notes that environmental variables cannot be fully controlled in practice and repeated measurements carry noise; a searching algorithm may also pick a suboptimal solution in the real world, because heuristics used during searching can remove candidates early and because profiling a kernel in isolation differs from running it in the complete graph.