Modular benchmark decouples quantum search-direction estimation from classical update rules, finding the best update rule depends on estimator and workload
Synopsis
The work presents a modular benchmarking framework that independently pairs search-direction estimators for parameterized quantum circuits (parameter-shift, finite differences, SPSA, PGPE, observable-curvature preconditioning) with classical update rules (SGD, Adam, RMSprop), evaluating them on MaxCut QAOA, Iris classification, a binary-MNIST QCNN, and molecular-hydrogen VQE under 256-shot finite-sampling simulation plus selected case studies on a 156-qubit processor; results show the preferred update rule depends on the estimator and workload, and that peak and across-seed average performance often favor different configurations.
Figure 1: Overview of the benchmarking framework. Four parameterized-quantum-circuit workloads are evaluated using a modular optimization loop that separates search-direction estimation from classical parameter updating. Experiments are conducted under finite-shot simulation and selected physical-QPU execution, with comparisons based on solution quality, quantum-evaluation cost, convergence behavior, and hardware-case-study outcomes. Derivative-free methods retain their native proposal and update dynamics
arXivInterpretation
The framework explicitly separates quantum search-direction estimation from classical parameter-update rules so their interaction can be examined independently. Prior comparisons treat each optimizer as a complete package, revealing which implementation performs well but offering less insight into why behavior changes across settings; this work pairs estimators and update rules separately. Four workloads evaluated under fixed hyperparameters, 256 shots, and seeds 42/123/1234, with a component-level comparison table.
Holding the estimator fixed, changing the update rule substantially changes outcomes, and the best rule depends on estimator and workload. Neither the update rule nor the direction estimator alone determines performance; selection should reflect task metric, sensitivity across seeds, and evaluation cost. Table 3 reports three-seed means and ranges for each estimator under SGD/Adam/RMSprop; adaptive rules consistently exceed SGD on classification, while energy tasks require estimator-specific choices.
Peak performance and across-seed average performance often point to different configurations. Reporting only a single best run can obscure differences in consistency across initializations. In MaxCut, SPSA-SGD and PSR-Adam share the single-run optimum while PGPE-SGD leads on average; in VQE, PGPE-SGD gives the closest single estimate while FiniteDiff-RMSprop minimizes mean absolute error.
Separating objective evaluations from logical circuit executions exposes costs introduced by parameter count, mini batches, shifted evaluations, and observable measurement. Equal optimization steps do not imply equal quantum-measurement budgets, so resource comparisons should report logical circuits and requested shots separately. Resource tables distinguish objective evaluations, logical circuits, and requested shots; for VQE, logical-circuit and shot totals are reported as not recorded because commuting measurement groups were not retained.
Perspective
The framework targets researchers and engineers who train parameterized quantum circuits under limited evaluation budgets, and applies to variational quantum algorithm workloads such as combinatorial optimization, supervised quantum machine learning, and quantum chemistry. Its modular pairing and resource-accounting convention can be reused to design new comparison experiments and support reporting peak results alongside across-seed averages. The hardware case studies address selected configurations to complement simulation findings, not as an end-to-end solver comparison or a test of quantum advantage.
Cross-task trends are associations rather than controlled effects of dimensionality, because parameter count, architecture, and task change together. Classification seeds also change the data partition and stochastic training sequence, so differences cannot be attributed exclusively to parameter initialization. Each hardware run is a single trajectory, which does not support confidence intervals or across-seed rankings and cannot attribute deviations to a particular gate, readout, or calibration effect. For VQE, commuting measurement groups were not retained, so objective evaluations cannot be converted into logical-circuit or shot budgets. Selecting a minimum under finite sampling can favor downward fluctuations, and agreement with chemical accuracy requires an independent precision assessment.