Sepsis glucose digital twins stop waiting for error thresholds: a frozen PatchTST plus reinforcement-learning residual fix improved one-step forecasts in all 10 ICU patients
Synopsis
Glucose digital twins in septic intensive care previously refit the model only after forecast error crossed a threshold, a reactive correction. This work reports a proactive workflow: a frozen PatchTST foundation model produces a one-step forecast, and an Actor–Critic reinforcement-learning agent uses the last 30-minute glucose window and recent residual history to apply a correction bounded at ±0.05 to that first step, which is then rolled out recursively over 15 steps. On minute-by-minute continuous glucose monitoring from 10 septic ICU patients, the pretrained agent lowered one-step forecast error in all 10 without patient-specific retraining, and mean absolute percentage error on the source patient's test set fell from 0.49% to 0.34%.
Interpretation
The update mechanism of a glucose digital twin moves from refitting after an error threshold is crossed to a bounded correction at every prediction cycle, with model parameters held fixed during deployment. The earlier digital twin workflow fine-tuned PatchTST on patient data only when ensemble relative error exceeded a 5% threshold, recalibrating after the discrepancy had grown large enough. On the source patient's test set, adding the reinforcement-learning correction lowered one-step mean absolute percentage error from 0.49% to 0.34%.
The correction comes from an Actor–Critic agent that reads both the recent glucose trajectory and recent prediction residuals and outputs a continuous action added to the baseline forecast. The agent's state combines a 30-minute look-back window with residual history, the action is clipped to ±0.05 before use, and the reward penalizes correction magnitude in addition to squared error. On minute-by-minute continuous glucose monitoring from 10 septic ICU patients, the pretrained agent without patient-specific retraining gave lower mean absolute percentage error than the uncorrected baseline in all 10, with relative improvements from 3.80% for S007 to 30.92% for S008.
In the hybrid workflow that stacks residual correction on threshold-triggered fine-tuning, most of the gain still comes from the correction layer rather than backbone updating. The hybrid lowered trajectory-averaged ensemble relative error in 9 of 10 patients and maximum ensemble relative error in all 10. Only S001 and S005 triggered outer-loop fine-tuning, and there fine-tuning added no improvement beyond reinforcement-learning correction alone, with S005 worsening; the reinforcement-learning-only version lowered trajectory-averaged ensemble relative error in all 10, with relative improvements of 0.87% to 10.02%.
Perspective
The demonstrated setting is minute-by-minute continuous glucose monitoring from 10 septic ICU patients, with both the foundation model and the reinforcement-learning policy trained on glucose series alone and evaluated by one-step and 15-step mean absolute percentage error and ensemble relative error. The authors propose larger multi-center validation and incorporation of clinical covariates such as nutrition, vasopressors and corticosteroids as next steps.
The authors state that the cohort is too small to cover the full range of ICU populations, illness severity, treatment protocols and glucose variability patterns. The correction acts only on the first step, with the remaining 14 steps rolled out recursively by the frozen model, which may limit handling of error accumulation deeper in the horizon; they also note that evaluation uses mean absolute percentage error and cumulative reward, without assessing reliability in hypo- and hyperglycemic ranges, uncertainty calibration, or robustness to missing or noisy sensor measurements.