Beijing Academy of Science and Technology

The authors introduce the AutoRegressive Retriever (ARR), a multimodal retrieval model that alternates between retrieving an item and updating the query embedding, appending each selected item's multimodal content to the query history before re-encoding and using the final embedding to rank the collection; training has two stages, supervised fine-tuning with stepwise contrastive supervision to teach feedback interpretation, then reinforcement learning that treats feedback items as actions and optimizes their selection using the final reciprocal rank of a relevant item, with a query-side adapter enabling this against a fixed item index.

The authors introduce the AutoRegressive Retriever (ARR), a multimodal retrieval model that alternates between retrieving an item and updating the query embedding, appending each selected item's multimodal content to the query history before re-encoding and using the final embedding to rank the collection; training has two stages, supervised fine-tuning with stepwise contrastive supervision to teach feedback interpretation, then reinforcement learning that treats feedback items as actions and optimizes their selection using the final reciprocal rank of a relevant item, with a query-side adapter enabling this against a fixed item index.