Autoregressive differentiable method for 0-1 integer programming beats open-source solvers on dense quadratic knapsack instances up to 10,000 variables

It casts solving a 0-1 integer program as autoregressive next-bit prediction: an arbitrary order of binary variables is fixed, and a transformer is trained to predict the next bit while staying in the feasible set. Rather than relaxing or regressing the whole solution vector at once, feasibility is imposed as a constraint on the generation process, so each generated step remains in the feasible region. The abstract describes the method design but does not give network size, training data volume, or implementation details of the per-bit feasibility constraint.

Training proceeds in two stages: first on feasible incumbents provided by any solver, which initializes the transformer inside the feasible set; then a Lagrangian penalty is applied to penalize infeasible solutions and the transformer is further trained with Gumbel-softmax activations on the relaxed objective to explore the feasible set. Solver-produced feasible solutions serve as an initialization signal, and a penalty term plus a differentiable relaxation drive exploration, forming a search-style training procedure that starts from a feasible point rather than relying on supervised imitation alone. The abstract describes the penalty and the use of Gumbel-softmax, but gives no penalty coefficients, training epochs, or ablation data.

On non-convex quadratic knapsack instances, the method shows consistent improvement over state-of-the-art open-source solvers for dense problems up to 10,000 binary variables. It extends validation of differentiable autoregressive solving to dense non-convex instances at the ten-thousand-variable scale and reports sustained gains relative to open-source solvers. The abstract reports the problem type, density, scale ceiling, and the comparative conclusion, but does not name the solvers, instance counts, time budgets, or improvement magnitudes.

It empirically demonstrates a phenomenon akin to a tunneling effect: the effective change of variables from binary variables to the continuous weights of the transformer lets the method cross barriers in the relaxed objective landscape. It attributes the performance gain to a landscape-crossing capability arising from the variable substitution, offering a mechanistic perspective on differentiable autoregressive solving. The abstract calls this an empirical demonstration and provides no visualization, barrier metric, or controlled experiment quantifying the effect.

The result targets 0-1 integer programs, especially dense instances such as non-convex quadratic knapsack, with the scale ceiling reported in the abstract as 10,000 binary variables; the method needs a feasible solution from any solver as its initialization source and relies on a Lagrangian penalty and Gumbel-softmax relaxation for subsequent training. For researchers and practitioners who want to plug learned methods into existing solving pipelines, this framework offers a reference path that starts from a feasible point and explores the feasible set in a differentiable autoregressive manner; its intended setting is problems with binary variables, definable feasibility constraints, and access to an initial feasible solution.

The abstract does not name the solvers, instance counts, time budgets, or improvement magnitudes, so the quantitative degree of the reported consistent improvement still needs confirmation from the main text; the tunneling effect is currently an empirical observation whose measurement and reproducibility await the main text; how strongly the method depends on the initial feasible solution, and how it performs on other 0-1 integer programs, are directions a careful reader can keep watching. Because this assessment is based only on the abstract, figures and experimental details are not included, and the quantitative questions above are open questions rather than conclusions.