Depression screening gets more than a label: InertiaMPDD measures departure from a healthy reference and corrects each decision
Synopsis
InertiaMPDD extends depression screening beyond a single recording-level label to a measurable departure from a healthy reference: a latent predictor of the next state is trained on frozen multimodal representations, and three descriptors are read from its prediction residuals, namely reference deviation, residual decay rate and a Jacobian spectral radius, used either for standalone screening or as an individual correction to a population decision. Across five cohorts from four datasets, the fused pathway reaches 98.41 F1 on D-Vlog and 84.98 F1 on LMVD, while on AVEC2014 the reference descriptors discriminate weakly and fusion reaches 66.67 F1.
Interpretation
Depression screening gains an interpretable intermediate quantity: how far the observed behavioral continuation in a recording departs from the next state predicted by a healthy reference. Earlier multimodal methods mainly predict recording-level depression labels directly, with limited evidence of comparison against a non-depressed reference; depression datasets commonly assign one label to an entire recording or participant without annotating emotional segments. On D-Vlog the population screening pathway NSP alone reaches 97.38 F1, and fusion with the reference-descriptor correction reaches 98.41; on LMVD, NSP reaches 74.57, the reference-only pathway IIP reaches 75.88, and fusion reaches 84.98.
The reference comes from a next-state predictor over frozen pretrained representations, and the gap between prediction and the actual next observation is compressed into three recording-level scalars. The three are reference deviation (average predictive disagreement), residual decay rate (persistence of residual fluctuations, estimated as a no-intercept first-order autoregressive coefficient shrunk toward a healthy-training prior) and the Jacobian spectral radius (local mapping strength of the predictor at the mean state, estimated with 30 Jacobian-vector power iterations). On LMVD full IIP reaches 75.88 F1, above 70.36 for a static MLP, 67.82 for a persistence baseline and 65.50 for a linear AR predictor; on MPDD-elder full IIP reaches 92.31, the static MLP 66.67 and the linear AR predictor 53.33.
The value of reference evidence and the benefit of fusion are separate things, and the same descriptors act differently across cohorts. On MPDD, standalone screening with full IIP reaches 92.31 F1 on the elder cohort and 90.91 on the young cohort, above 85.71 and 80.00 for fixed additive fusion; on AVEC2014 fusion reaches 66.67 F1, an improvement over 50.00 for either pathway alone but still below the largest displayed baseline value. On AVEC2014 the depressed-minus-healthy residual contrast reverses direction between the training and test splits, and neither is significant; the test-only spectral contrast is also not significant, and spectral-only screening is not significant either.
Perspective
The demonstrated scope is five cohorts from four datasets (D-Vlog, LMVD, AVEC2014, MPDD), where D-Vlog and LMVD are self-recorded vlogs and AVEC2014 and MPDD are interview or task-elicited recordings; AVEC2014 uses a subject-isolated split of 83 participants (67 training, 16 test), and MPDD is retrained per cohort. The authors name reference coverage as a key boundary and state that a fixed reference on an unseen cohort requires a separate transfer test.
The authors call the three quantities candidate proxy biomarkers whose association with depression labels is established through screening rather than direct emotion annotation; stress, fatigue and acquisition conditions may also affect predictive departure. They report weak reference discrimination on AVEC2014 and a residual contrast between depressed and healthy groups that reverses direction between training and test splits; the Jacobian spectral estimate depends on fixed learned coordinates, and the mean state need not be an equilibrium. Specificity, repeatability and transfer still require further validation.