Erik Müller

For stochastic-gradient optimization with very many inequality constraints, an adaptive constraint-sampling method stops drawing each step's constraint uniformly from all constraints and instead maintains an "important set" of recently violated constraints, sampling from that set or from all constraints with fixed probabilities. On 100-dimensional synthetic convex problems it lowers the final Euclidean distance of the iterates to the constrained minimizer by about two orders of magnitude on average for ABSGD, Projection-SGD and Huber-penalty SGD, and by one to two orders for APDSG; in Dubins Car control-barrier-function synthesis it lowers maximum constraint violation by half an order to one order of magnitude.

For stochastic-gradient optimization with very many inequality constraints, an adaptive constraint-sampling method stops drawing each step's constraint uniformly from all constraints and instead maintains an "important set" of recently violated constraints, sampling from that set or from all constraints with fixed probabilities. On 100-dimensional synthetic convex problems it lowers the final Euclidean distance of the iterates to the constrained minimizer by about two orders of magnitude on average for ABSGD, Projection-SGD and Huber-penalty SGD, and by one to two orders for APDSG; in Dubins Car control-barrier-function synthesis it lowers maximum constraint violation by half an order to one order of magnitude.