Wrist pulse waveforms carry meal information: aligning them with food text wins 0.69 of meals and lifts calorie-intake AUC by 11%

Wrist PPG carries meal-specific information around meals that a language model can read out, beyond participant identity or time of day. PPG had already been used to predict cardiovascular, metabolic and biological-clock information, but whether passively collected wrist PPG contains postprandial physiological responses had not been tested. On held-out meals, near-meal PPG (the segment closest to the meal within 4 hours either side) reached a meal-description win rate of 0.69 (95% CI 0.67–0.70); a within-subject control matched for time of day but with shuffled meal pairings reached 0.60 (0.57–0.63), and across-subject shuffling reached 0.50 (0.47–0.53), with non-overlapping confidence intervals.

The aligned representation transfers to dietary tasks and predicts postprandial satiety change in an independent cohort without retraining. Text-only baselines scored lower on the same tasks, and a text baseline augmented with heart rate, heart-rate variability and time-of-day covariates still fell below the multimodal model. In the Apple Heart & Movement Study, AUC for separating days above versus below a person's mean intake was 0.82 (0.814–0.826) versus 0.74 (0.732–0.748) for text only and 0.76 (0.749–0.764) for text plus covariates; in the 140-participant, 720-meal Validation Study, satiety AUC was 0.71 (0.64–0.78) versus 0.64 (0.57–0.71) for text only, a paired ΔAUC of +0.07 (0.01–0.13, p = 0.03).

The method maps one PPG segment through a linear adapter into two pseudo-tokens placed before a frozen GPT-2 language model, which then generates meal descriptions likely eaten near that time. Both the PPG encoder and the language model stay frozen, so only the linear projection from the 256-dimensional PPG embedding into language-model token space is trained, making the alignment an interface between the two modalities. Training used 1,122,834 self-reported meal descriptions from 19,340 participants, split 80/20 by participant into training and held-out test; the Validation Study data were held out entirely for external testing with no fine-tuning.

When meal descriptions are shortened or condensed by a large language model into a single keyword, PPG recovers part of the lost information. Text-only AUC falls as text is removed, while the multimodal model still beats the full-text-only baseline with 50% of the text removed. With descriptions condensed to one representative word by Gemma 27b, NPLM reached AUC 0.75 (0.74–0.76), above the full-text-only 0.74, a paired ΔAUC of +0.010 (0.001–0.019, p = 0.03).

The findings rest on two cohorts: the Apple Heart & Movement Study, where U.S. Apple Watch users self-reported meals and which supplied training data, and the Validation Study, a controlled dining-facility lunch observation of 140 participants and 720 meals used for external testing, with satiety assessed only at midday weekday meals. The authors propose prospective studies on metabolic endpoints such as weight management and glycemic control, and extension to laboratory biomarkers.

Meal logs and appetite ratings in both cohorts are self-reported, and the authors expect stronger results with controlled measurement; the 4-hour window around meals is broad and may include unlogged intake, physical activity or stress; the validation cohort differs demographically from the training cohort and was not matched, limiting subgroup inference; and without randomized or longitudinal intervention data, whether model-driven feedback could improve outcomes cannot be judged. The authors also state that the 11% AUC gain is a population-level discrimination gain on a binary task and should not be read as supporting standalone clinical prediction of caloric intake from PPG alone.