ARR lets a multimodal retriever iteratively rewrite its query from retrieved items, topping averages on M-BEIR and seven zero-shot benchmarks
The authors introduce the AutoRegressive Retriever (ARR), a multimodal retrieval model that alternates between retrieving an item and updating the query embedding, appending each selected item's multimodal content to the query history before re-encoding and using the final embedding to rank the collection; training has two stages, supervised fine-tuning with stepwise contrastive supervision to teach feedback interpretation, then reinforcement learning that treats feedback items as actions and optimizes their selection using the final reciprocal rank of a relevant item, with a query-side adapter enabling this against a fixed item index.