A few altered seconds in a long video can now be placed: separate audio and visual readouts lift AV-mAP from 84.76 to 91.30
Synopsis
Manipulations in long videos often touch only a few seconds, while existing audio-visual deepfake benchmarks mostly use short talking-head clips and long-form forensics has been largely visual. AV-FABLE pairs 16,650 pristine–manipulated twins (998 hours, 27 editing configurations) with separate audio and visual timelines, so localization must report both the time boundaries and the affected modality. Reading frozen Qwen3-Omni perception features through separate audio and visual temporal decoders, NFSR reaches 91.30 in-domain AV-mAP against 84.76 for UMMAFormer on the same features, and the highest AV-mAP in all three out-of-distribution settings.
Interpretation
Long-form forensics now has to answer which seconds were altered and whether the audio or the picture was altered, not just whether the whole video is fake. Earlier audio-visual deepfake benchmarks mostly target speaker-centered short clips, and long-form temporal forensics has been predominantly visual, with audio and visual sharing one timeline. AV-FABLE holds 16,650 pristine–manipulated twin pairs spanning 998 hours and 27 editing configurations, with manipulated intervals covering a median 9.45% of each manipulated video and separate audio and visual event records.
On the same frozen perception features, supervised readouts recover forensic signal that prompted outputs do not expose. Zero-shot prompting of Qwen2.5-Omni and Qwen3-Omni yields 51–52 detection AUROC on AV-FABLE and at most 0.30 mAP on either track. Linear probes on those frozen features separate manipulated from authentic segments at 90.8 audio and 80.8 visual AUROC, indicating the evidence is encoded but not surfaced through language outputs.
NFSR reads frozen audio and visual encoder features directly into manipulation intervals on two independent timelines. Against UMMAFormer, which shares the same frozen Qwen3 features and training budget, NFSR raises in-domain AV-mAP from 84.76 to 91.30, with the largest gain on the visual track. In-domain NFSR reaches 91.30 AV-mAP, 96.04 detection AUROC and 83.61 attribution F1, and records the highest AV-mAP in source transfer, held-out generators and held-out TI2V.
Perspective
The demonstrated scope is localization, detection, modality attribution and explanation of sparse audio-visual manipulations in open-scene long-form videos, evaluated on AV-FABLE's source-disjoint splits and three out-of-distribution settings with thresholds fixed on source-domain calibration data.
Explanation is the most challenging dimension, with current baselines producing well-formed outputs that often misidentify the operation or target; visual coverage under shift remains the main open problem, with NFSR's V-IoU between 26.42 and 41.21.