LACS moves the pruning cutoff with the current confidence spread: 29% fewer backward rows on CIFAR-10 at 89.77% versus 89.82% accuracy
Synopsis
Which examples receive backward computation during training no longer has to follow a preset quota or an absolute confidence cutoff. LACS compares each candidate's exponentiated maximum predictive confidence with the current population mean of exponentials, clips the coefficient at zero, and thereby applies a cutoff equal to a log-mean-exp reference plus a margin, with the retained count following the score distribution. On CIFAR-10 it removes 29% of backward rows relative to full cross-entropy at accuracy means of 89.77% versus 89.82%, with a three-seed interval crossing zero; on 14 tabular datasets quadratic boosting lowers mean test error from 23.36% to 22.96%.
Interpretation
Which examples enter backward propagation is now decided by the current batch's confidence distribution, which sets both the threshold and the retained count. A fixed quota fixes the amount removed before the score gaps are examined, and an absolute confidence cutoff ties the decision to the score scale; neither moves with the current distribution. On CIFAR-10 with ResNet-32, LACS removes 29% of backward rows relative to full cross-entropy, with accuracy means of 89.77% versus 89.82% and a confidence interval that crosses zero over three seeds.
The rule reads only maximum predictive confidence, so selection needs neither labels nor per-example gradients. Each candidate's exponentiated maximum confidence is compared with the current population mean of exponentials, and clipping the coefficient at zero yields a cutoff equal to a log-mean-exp reference plus a margin. Across 14 tabular datasets, quadratic weak learners lower mean test error from 23.36% to 22.96%, and full training replays use about 30% fewer fitting rows.
Saved computation is not the same as saved time, and the two are counted separately. The current neural implementation runs a no-gradient scoring forward to obtain fresh scores, then a forward and backward on retained examples only, so backward sparsity does not become wall-clock acceleration. On one host LACS is slower than full cross-entropy, while a 300-round RBF-SVM replay on credit data fits 29.3% fewer rows, cuts end-to-end time by 37.2%, and slightly lowers final error.
Perspective
The demonstrated scope is seven evaluation settings: supervised neural training, Transformer language modeling, long-tailed classification, semi-supervised learning, and boosting with compact nonlinear, tree and kernel-SVM weak learners; neural head-to-head comparisons are limited to CIFAR-10 and CIFAR-100 with three seeds each, tree comparisons use the fixed-threshold companion rule LACS-T, and the RBF-SVM timing evidence rests on four dataset/split cells with nearly equal round counts.
On CIFAR-10 the gap to InfoBatch-B is positive in all three seeds but statistically unresolved, and the comparison with InfoBatch is also unresolved; the current two-pass neural implementation is slower than both full cross-entropy and InfoBatch; on Transformer language modeling, selective execution reaches a validation NLL of 1.5688 versus 1.5666 while remaining 4.5% slower than the optimized dense implementation; long-tailed and semi-supervised settings provide boundaries rather than positive evidence.
