Skip to main content
Search

# Factored Spatio-Temporal Explanations

1 related articles

Subscribe to updates
  1. arXiv

    Video action explanations split into what the model sees and when it moves: lowest deletion AUC on most axes across three datasets, with masks about five times sparser

    Explanations for video action recognition usually fold spatial appearance and temporal motion into a single saliency map, so a user cannot tell whether the model predicted from the object or from the movement. This work separates the explanation into a spatial appearance mask and a temporal motion mask, optimized jointly against a frozen classifier, with nearest-frame replacement used to remove temporal dynamics. On EPIC-Kitchens-55, EGTEA Gaze+, and Something-Something V2, the method attains the lowest deletion AUC on most datasets and axes, with masks roughly one-fifth the size of the strongest baseline.

You’re all caught up