Tabular regression objectives get a fresh ranking: quantile loss is steadier on four backbones, while early stopping stalls between noise regions

The choice of objective for tabular deep regression is re-compared systematically, and quantile loss gives more consistent test-RMSE gains over a tuned mean squared error baseline across several backbones. Alternative objectives had been evaluated on different datasets, backbones and tuning protocols, making it hard to tell which alternatives reliably beat mean squared error. On 41 BeyondArena tabular datasets under one protocol with tuning and retraining across 15 random seeds, quantile loss gains over mean squared error are +0.55%, +0.46%, +0.40% and +0.58% on MLP, TabPack, RealMLP and TabM.

Samples with different target-noise levels reach their best predictive performance at different training epochs, so a single global early-stopping checkpoint imposes a compromise between those regions. Standard global early stopping selects one checkpoint by average validation error and applies it uniformly to all inputs. On synthetic data with noise added only to training targets and noise-free validation and test targets, selecting checkpoints per estimated-uncertainty cohort beats random cohorts on the heteroskedastic variant, while the homoskedastic control gets worse.

An extension of quantile loss with input-dependent smoothing improves over standard quantile loss on several backbones and adds no test-time cost. Standard quantile loss can represent heteroskedastic target distributions but does not address the compromise a shared stopping point creates across regions. On 28 datasets with at least 1,000 training samples, the extension gains +0.66%, +0.25%, +0.82% and +0.97% over mean squared error on MLP, TabPack, RealMLP and TabM, above the quantile-loss gains of +0.47%, +0.22%, +0.35% and +0.39% in the same set of experiments.

The comparison covers conventional tabular DNN backbones; pretrained PFNs are outside its scope. Evaluation uses 41 BeyondArena datasets, excluding three computationally demanding ones, and the main AdaQR comparison is further limited to 28 datasets with at least 1,000 training samples. Code and experiment configurations are available in a public repository, whose README says it can be used to download the datasets and reproduce the experiments.

The authors state that understanding of the underlying training dynamics remains incomplete, and that AdaQR is one possible remedy whose benefits vary across backbones and datasets. On real datasets, cohort-based checkpoint selection yields smaller improvements for AdaQR than for standard quantile regression. AdaQR's theoretical characterization assumes Gaussian label noise, Gaussian predictions, sufficient quantile resolution and convergence to the Bayes optimum, and these assumptions are a theoretical framework rather than strict prerequisites for empirical success.