VR football viewers get a four-channel immersion readout: video, audio, EEG and gaze fused to 81.6% four-level accuracy
Synopsis
Judging how immersed a VR football viewer is at a given moment is hard from pictures alone, since video cannot tell watching from disengagement, and naive concatenation of four signals carries low-quality channel noise into the decision. CMAFNet treats video as a semantic hub, runs bidirectional cross-attention with audio, EEG and eye tracking, weights each modality per sample by real-time signal quality, and uses an ordinal regression loss to keep immersion levels ordered. On 86,400 samples cut from 720 sixty-second VR match clips, four-level classification reaches 81.6% overall accuracy and 0.274 mean absolute error, 3.4 points above and 13.8% below the strongest 2024 baseline, MT-AMW.
Interpretation
Immersive VR football broadcasting moves from post-hoc questionnaires to real-time immersion-level inference from synchronized video, audio, EEG and eye-tracking signals. Prior VR immersion detection focused on single tasks or limited modality combinations, and simple concatenation ignored semantic gaps and quality differences, letting noise from low-quality channels reach the final decision. On 86,400 temporal samples cut from 720 sixty-second VR match clips, four-level immersion classification reaches 81.6% overall accuracy and 0.274 mean absolute error, 3.4 percentage points above and 13.8% below the strongest 2024 baseline, MT-AMW.
The four signals are not fully paired; video interacts with each of the other three, and fusion weights are set per input sample. Fully connected cross-attention would need six pairwise attention modules for four modalities, whereas this design keeps three bidirectional paths, video with audio, EEG and eye tracking, cutting cross-attention parameters by 50%. When baselines were re-implemented with the same encoders and preprocessing, they gained only 0.5 to 0.8 percentage points, and CMAFNet still led the strongest re-implemented baseline, MT-AMW, by 2.9 points in accuracy and 13.3% in error.
Fusion weights follow signal quality rather than staying fixed across inputs. Earlier gated fusion such as CMGA learns one scalar weight per modality shared across all inputs, so it cannot track per-sample quality variation. Injecting Gaussian noise into a single modality at 0 dB SNR drops the EEG channel's mean gating value from 0.53 to 0.19, a 64% reduction, while the other three clean modalities stay roughly stable or rise slightly.
Leave-one-subject-out cross-validation across 30 participants lands close to the standard split. The 8:1:1 split already keeps participants disjoint, but few test participants may not characterize cross-individual generalization. Leave-one-subject-out cross-validation gives a mean accuracy of 79.3±2.1%, 2.3 points below the 81.6% standard split and above all eight baselines on the standard split.
Perspective
Results come from 30 adult volunteers aged 18 to 35 with some technical familiarity, recorded in a laboratory with a Meta Quest Pro headset and a 32-channel dry-electrode EEG cap, over four match scenario types.
The authors note that two probe targets are clip-level participant self-reports carrying Likert-scale noise and cannot resolve sub-clip information; unique-information magnitudes should be read as informative rather than precise; and the targeted perturbations establish association rather than strict causal attribution.