# Zeroth-Order Optimization

Vision models still fail on seemingly simple scenes, and a fixed dataset cannot reliably estimate worst-case error. Procedural adversarial learning treats an interpretable procedural generator as a controllable data source and uses zeroth-order optimization to search its low-dimensional control parameters for hard instances. On four tasks spanning surface normals, image segmentation and human pose estimation, Simple-PAT lowers worst-case error below uniform training and two robust reweighting baselines against a cross-model black-box adversary with 50M aggregate queries.

Vision models still fail on seemingly simple scenes, and a fixed dataset cannot reliably estimate worst-case error. Procedural adversarial learning treats an interpretable procedural generator as a controllable data source and uses zeroth-order optimization to search its low-dimensional control parameters for hard instances. On four tasks spanning surface normals, image segmentation and human pose estimation, Simple-PAT lowers worst-case error below uniform training and two robust reweighting baselines against a cross-model black-box adversary with 50M aggregate queries.

Zero-order fine-tuning previously treated training examples alike; SAZO learns each example's importance online and allocates queries accordingly, improving average downstream performance across four model families and cutting the queries needed to reach the same training loss by up to 6x.

Zero-order fine-tuning previously treated training examples alike; SAZO learns each example's importance online and allocates queries accordingly, improving average downstream performance across four model families and cutting the queries needed to reach the same training loss by up to 6x.

This paper proposes and analyzes Comparison-based Preference Optimization (ComPO), a zeroth-order alignment method based on comparison oracles that, instead of directly optimizing a differentiable preference loss, perturbs the current policy and judges whether each perturbation raises the likelihood of preferred responses and lowers that of dispreferred ones to extract an update direction, thereby exploiting low-margin "noisy" preference pairs; the authors establish a convergence guarantee for the basic offline scheme under smoothness, gradient sparsity, and oracle compatibility, introduce an online version that uses unlabeled policy generations for reverse-KL control, and prove a performance guarantee for a basic constrained scheme under local coverage and in-distribution pairwise reward

This paper proposes and analyzes Comparison-based Preference Optimization (ComPO), a zeroth-order alignment method based on comparison oracles that, instead of directly optimizing a differentiable preference loss, perturbs the current policy and judges whether each perturbation raises the likelihood of preferred responses and lowers that of dispreferred ones to extract an update direction, thereby exploiting low-margin "noisy" preference pairs; the authors establish a convergence guarantee for the basic offline scheme under smoothness, gradient sparsity, and oracle compatibility, introduce an online version that uses unlabeled policy generations for reverse-KL control, and prove a performance guarantee for a basic constrained scheme under local coverage and in-distribution pairwise reward