# MultiSHAP

Multimodal models answer questions and retrieve images, yet which image region and which word jointly push a given output score has been hard to quantify. MultiSHAP applies the Shapley Interaction Index from cooperative game theory to estimate, for every image-patch and text-token pair, a second-order synergy or suppression score, using only a query interface that allows separate perturbation of image and text and returns a task-relevant scalar. On 500 VQAv2 samples, deleting patches in the resulting ranking lowered the target-answer probability by 0.1720 on average, versus 0.0032 for the lowest-ranked patches.

Multimodal models answer questions and retrieve images, yet which image region and which word jointly push a given output score has been hard to quantify. MultiSHAP applies the Shapley Interaction Index from cooperative game theory to estimate, for every image-patch and text-token pair, a second-order synergy or suppression score, using only a query interface that allows separate perturbation of image and text and returns a task-relevant scalar. On 500 VQAv2 samples, deleting patches in the resulting ranking lowered the target-answer probability by 0.1720 on average, versus 0.0032 for the lowest-ranked patches.