Streaming video agents stop recording only what is present: DeltaStreamer logs frame-to-frame changes and scores 66.07% on OVO-Bench
Synopsis
Streaming video understanding must retain evidence before anyone knows which details later queries will need, yet existing methods flatten event transitions when compressing history and cannot recover details once a frame has passed. DeltaStreamer explicitly records visual changes between adjacent frames in a hierarchical textual memory, and after a query arrives a query-driven planner decides whether to retrieve, keep observing, or replay historical frames. It scores 66.07% overall on OVO-Bench and 59.53% on StreamingBench, above the representative streaming methods it is compared with.
Interpretation
A streaming video system can now write down how events change during observation, not only what is present in the scene. Earlier streaming methods managed the budget through token compression, KV-cache reuse and multi-turn dialogue, so their records captured what is present while leaving inter-frame change implicit. On OVO-Bench, with identical two-frame input, 2F-NoDelta without change recording scores 61.12 and 2F-VisualDelta with visual comparison scores 66.07.
Once a question arrives, the system first decides what kind of evidence is needed and whether it lies in the past or the future, then retrieves memory or revisits original frames accordingly. The planner decomposes the query into target and temporal anchor, evidence type, temporal scope and search strategy, and keeps that plan available through later reasoning. On OVO-Bench, adding plan generation raises the overall score from 59.30 to 63.85, persistence raises it to 64.77, and adding the evidence contract reaches 66.07.
Pixel-level details missing from textual memory can be recovered by following source links back to historical frames. Once earlier records were written, evidence was limited to the retained visual signal, with no route back to the original frame. In the OVO-Bench tool ablation, removing replay lowers the overall score to 63.28, the largest drop among the tool removals.
Perspective
The demonstrated scope is two streaming video question-answering benchmarks, OVO-Bench and StreamingBench, using Qwen3.6-27B as the shared backbone for observation, planning and answering, sampling at 1 fps and running BF16 inference on 8 NVIDIA A100 GPUs.
Replay requires extra frame access and visual processing and may increase response latency; visible memory stays compact, but records and historical frames still cost storage. Errors in observations or temporal anchors can affect retrieval, and agreement across memory layers does not guarantee a correct answer. Proactive output (PO) on StreamingBench is only 10.40, showing that better evidence handling alone does not solve response timing.