# Alternative Annotator Test (alt-test)

The authors built MIST (the Misleading-Image Stress Test), 200 English sentences each containing a phrase readable either figuratively or literally and shown with an aligned image, a misleading image, or no image; across thirteen VLM judges an aligned image changed 20.5% of labels and a misleading one 19.4%, versus 11.6% when only the ignore-the-image instruction was deleted with the image left in place, and only 37% of the labels that differ between the two images moved toward the sense shown, while agreement with the human annotators was unchanged whether the image was absent, aligned, or misleading.

The authors built MIST (the Misleading-Image Stress Test), 200 English sentences each containing a phrase readable either figuratively or literally and shown with an aligned image, a misleading image, or no image; across thirteen VLM judges an aligned image changed 20.5% of labels and a misleading one 19.4%, versus 11.6% when only the ignore-the-image instruction was deleted with the image left in place, and only 37% of the labels that differ between the two images moved toward the sense shown, while agreement with the human annotators was unchanged whether the image was absent, aligned, or misleading.