Neural processes assumed clean inputs: DnNP folds denoising into meta-learning and cuts PM2.5 interpolation RMSE by up to 30%
Synopsis
Standard neural processes treat inputs as noise-free observations, so sensor measurement error degrades both predictions and uncertainty estimates. DnNP makes the clean input a latent variable inside the neural-process framework and trains a denoiser jointly with meta-learning through variational inference, predicting directly from noisy inputs. On PM2.5 spatial interpolation from 34 air-quality sensors in Brookline, Massachusetts, DnNP reduces RMSE by up to 30% and NLL by up to 73% relative to standard neural processes.
Interpretation
The neural-process family gains a member that handles input noise explicitly: DnNP no longer treats noisy coordinates as ground truth, but models the clean input as a latent variable alongside function-level uncertainty. Earlier neural-process variants, including conditional, latent, attentive and convolutional conditional neural processes, assume noise-free inputs; prior work addressed label noise, and training neural processes on noise-contaminated inputs had not been addressed. On PM2.5 spatial interpolation from 34 QuantAQ sensors in Brookline, Massachusetts, over May 24 to September 19, 2024, DnNP beats standard neural processes in NLL and RMSE across three feature configurations, with RMSE reduced by up to 30% and NLL by up to 73%.
The training objective puts denoising and prediction into one variational lower bound: a predictive reconstruction term scores under denoised inputs, a measurement-consistency term ties the denoiser to the known noise model, and two KL regularizers constrain the denoiser and the latent encoder. The denoiser is a small multilayer perceptron applied per input that infers clean inputs from noisy measurements, and the latent encoder then works on denoised inputs; at test time the model samples clean inputs, then the function-level latent, then predicts each target. Evaluation covers 1D Gaussian-process regression, MNIST image completion and two real spatial-interpolation datasets; on MNIST, DnNP beats a standard neural process trained on noisy inputs in NLL and RMSE at every noise level and approaches the noise-free baseline at low to moderate noise.
Perspective
The demonstrated setting is input noise independent across observations, covering 1D Gaussian-process regression, MNIST image completion, PM2.5 spatial interpolation in Brookline, Massachusetts, and sea-level pressure interpolation across NOAA stations in California; the method requires a user-specified noise model and prior over clean inputs.
Spatially correlated sensor bias is not modeled, since the denoiser acts pointwise per observation; the noise model and prior reduce the purely data-driven character of neural processes, although the method stays robust to moderate misspecification of the noise level. In the (Lat, Lon, Temp) setting for California sea-level pressure, DnNP and the standard neural process are statistically tied in NLL while DnNP still lowers RMSE.