Stress recognition in therapy-style dialogue stops reading one sentence at a time: an emotion-transition matrix lifts five-class accuracy to 85%
Recognition of psychological stress in dialogue has mostly treated each utterance as an isolated fragment and concatenated modality features at the final layer, which cannot capture how emotion accumulates and shifts across turns. BERT-EmoShiftNet pairs BERT's multi-level semantic encoding with EmoShiftNet's emotion-transition modelling, aligns text, speech and vision through a cross-modal Transformer, and unifies them with cross-modal contrastive learning in a shared embedding space. On a dialogue-level split of a self-built dataset, full-modal five-class accuracy is 85.0% (95% CI [83.2%, 86.8%]), emotion-transition prediction accuracy is 76.2%, and mean detection latency is 13.4 s.