Beyond whether an image is AI: EviScene checks the depicted situation, reaching 52.26% evidence sufficiency
Synopsis
Image factuality assessment no longer stops at whether a picture is AI-generated: the EviScene multimodal agent refines its own questions during investigation and reaches 79.05% balanced accuracy and 52.26% evidence sufficiency on 1,527 test images.
Interpretation
Image Factuality Assessment is posed as its own task: given an image and a generic verification instruction, a system returns a supported or refuted verdict with an evidence-grounded explanation. The earlier question stopped at whether an image is AI-generated, which does not establish whether the depicted situation holds. The EviLens benchmark contains 10,174 supported and refuted images from controlled construction and web collection, covers twelve factual error types, and evaluates both verdict accuracy and evidence sufficiency.
EviScene interleaves visual inspection, external retrieval, and evidence analysis, using new findings to refine its questions and revisit relevant image details. It is a multimodal ReAct agent, meaning an agent loop that alternates reasoning with actions, rather than a one-shot verdict. A compact policy is first supervised-fine-tuned on reviewed teacher trajectories, then given privileged on-policy self-distillation over verified decisions from the policy's own investigations.
On the 1,527-image test set, the trained Qwen3.5-9B agent reaches 79.05% balanced accuracy and a Strict Evidence Sufficiency Rate of 52.26%. The same-backbone base agent is 6.71 and 9.37 percentage points lower on those two measures. Its evidence sufficiency rate is the highest among the evaluated agents.
Perspective
The work targets verification settings where a reader must decide whether a depicted situation is factually supported, and it addresses researchers in image forensics, fact-checking, and multimodal agents. Evaluation runs on 1,527 test images from the EviLens benchmark, spanning controlled construction and web collection. Code, models, and data are planned for release on GitHub and Hugging Face.
An evidence sufficiency rate of 52.26% shows that a correct verdict and sufficient supporting evidence remain separate matters, and in most cases evidence sufficiency does not yet meet the strict standard. Twelve factual error types and two image sources define the current evaluation scope, and whether other error types and sources behave the same way is not stated.