High-dimensional asymptotics yield a criterion for deciding whether private external datasets are worth buying using only summary statistics
Related research and updates
Synopsis
The work models dataset selection as high-dimensional regression with multiple heterogeneous sources and a weighted ridge estimator, uses only summary statistics, and provides privacy guarantees either on labels only or jointly on features and labels in terms of rho-zero-concentrated differential privacy; its main technical contribution is a deterministic equivalent of the test error that captures interactions among sample size, covariance structure, model shift, regularization, and privacy noise, allowing hyperparameters (weights and ridge regularizers) to be optimized and the usefulness of private external datasets to be judged without accessing the data itself, supported by experiments on synthetic and real-world datasets.
Figure 1: Dependence of target excess risk on covariance shift, model shift and privacy noise in a synthetic linear regression experiment with two clients. We take x1 ∼N(0, Σ1) and x2 ∼N(0, Σ2), with Σ1 and Σ2 of the form [K(ρ)]ij = ρ|i−j|. We set Σ1 = K(ρ1) with ρ1 = 0.1 and Σ2 = a2K(ρ2), so that Tr(Σ2Σ−1
arXiv · Page 7Interpretation
The paper provides a deterministic equivalent of the test error that brings sample size, covariance structure, model shift, regularization, and privacy noise into a single expression. The abstract presents this deterministic equivalent as the main technical contribution and states that it enables hyperparameter optimization and dataset decisions from population-level quantities. Evidence comes from the abstract's statement of the theoretical result and from the authors' experiments on synthetic and real-world datasets; the abstract gives no specific error values or experimental scale.
The method relies only on publicly available summary statistics rather than individual-level data to decide whether to adopt external data. The abstract lists using only summary statistics as the response to the challenge that decisions often rely on aggregated statistics, and states that the usefulness of private external datasets can be decided without accessing the data itself. Based on the abstract's description of the method's setting; the abstract does not list the specific statistics used or estimation error bounds.
Privacy guarantees are given in terms of rho-zero-concentrated differential privacy, covering both the labels-only and the joint features-and-labels cases. The abstract explicitly distinguishes the two privacy scopes and notes that privacy noise can offset the benefit of a larger sample size, a trade-off incorporated into the theory. Evidence is the abstract's statement of the privacy definition; the abstract gives no specific rho values or details of the noise-injection mechanism.
The theory is positioned as a tractable foundation for private transfer learning and is accompanied by experiments on synthetic and real-world datasets. The abstract connects the dataset selection problem to a private transfer learning framework, stating that the goal is to provide a theoretical basis for decisions about buying external data or participating in collaborative learning. Evidence is the abstract's statements about experimental support and theoretical positioning; the abstract reports no dataset names, sample sizes, or comparison baselines.
Perspective
The result targets researchers and practitioners who must decide whether to bring in external data under privacy constraints; the applicable setting is high-dimensional regression with multiple heterogeneous sources described by publicly available summary statistics and a weighted ridge estimator. In that setting, the theory can be used to choose weights and ridge regularization hyperparameters and to judge whether private external datasets are worth adopting. The privacy guarantees described in the abstract cover labels-only and joint features-and-labels scopes, so the applicable scope varies with what is being privatized.
The abstract does not give the concrete form of the deterministic equivalent, privacy parameter values, the datasets or sample sizes used in experiments, or comparisons against alternative selection criteria; the quantitative trade-off between negative transfer and privacy noise remains an open question at the abstract level. In addition, the reading scope here is the abstract only, so figures and experimental details in the body are not included, and the judgments above are limited to what the abstract states.