Wen Zhang

Retrieval quality in dynamic product advertising could previously be compared only through sparse, biased engagement logs or costly online experiments. The new method compresses a user's engagement history into a small set of intent representatives grouped by semantic ID and stratified by event type, then has a large language model judge each candidate product against every intent and aggregate the results into a user-level score. On 30 days of production traffic, the score ranked six retrieval sources consistently with conversion precision and recall, and moved in the same direction as online CTR and CVR across four A/B tests.

Retrieval quality in dynamic product advertising could previously be compared only through sparse, biased engagement logs or costly online experiments. The new method compresses a user's engagement history into a small set of intent representatives grouped by semantic ID and stratified by event type, then has a large language model judge each candidate product against every intent and aggregate the results into a user-level score. On 30 days of production traffic, the score ranked six retrieval sources consistently with conversion precision and recall, and moved in the same direction as online CTR and CVR across four A/B tests.