Retail media ads stop counting clicks alone: theory-anchored semi-Markov states lift one-step purchase AUC from 0.635 to 0.717

Retail media targeting shifts from static transaction summaries to inference over purchase-intention states that change over time in the clickstream, and that inference selects the next ad. Conventional targeting summarizes behavior through aggregate features such as click-through rate, conversion rate, recency, frequency, monetary value, dwell time and basket activity; these summaries compress away temporal structure, so two shoppers with the same recent event counts can differ in transition pressure and state duration. On the anonymized omni-channel retailer's platform, a one-step leakage-free paired holdout comparison under the same data split gives AUC 0.717 for the theory-identified model versus 0.635 for the action-only covariate baseline, with DeLong Z=5.64 and p=1.7×10⁻⁸ on n=1,008 paired test consumers.

The latent states are not statistical clusters learned freely from click sequences; their semantics are declared from consumer-behavior theory first, and their dynamics are then fitted to data. A hidden semi-Markov chain learned from clickstream alone is generically under-identified: different initial, transition, emission and duration parameters can produce nearly indistinguishable click-sequence distributions while inducing different latent trajectories. The state space is the Cartesian product of a Consumer Decision Journey stage, a Theory of Planned Behavior intention-strength coordinate and a Persuasion Knowledge Model trust stance, with the 24 composite states of highest empirical relevance retained; in the ablation, a data-driven hidden semi-Markov model at the same K=24 but without theory coordinates scores AUC 0.633, statistically indistinguishable from the 0.635 baseline (p=0.58).

The retrieval-augmented language model supplies an auditable reference posterior rather than a standalone predictor. A language model alone provides no generative model of click sequences and no duration distribution; each call is a conditional judgment on the current context. Feeding the language model's per-key marginal to a non-sequential classifier drops AUC to 0.508, below the 0.635 baseline (Z=−2.93, p=0.0033); setting the mixing weight to λT=0 and removing the KL constraint gives AUC 0.673, still above baseline but 4.4 percentage points below the full model.

Deployment-side evidence comes from three sequential campaign waves, not a simultaneous controlled causal experiment. The first wave built the model and strategy, the second tested whether the 15 covariate cells flagged in the first wave kept their direction in a fresh non-overlapping cohort, and the third re-validated the selected policy over a later time window. In the second wave 13 of 15 cells kept their direction, a cell-level directional agreement of 86.67%, with 80.65% verified checkout-direction agreement; the third wave raised checkouts from 358 to 512 relative to the first, a temporal before-after audit.

The demonstrated scope is one anonymized East Asian omni-channel convenience-store retailer's retail media platform, covering 5.2 million active loyalty members, with an event table holding anonymized member identifier, session identifier, event time, click symbol, shopper trait, product-involvement tag, ad-type tag and campaign identifier; the three campaigns ran August 4–17, 2023, November 24–December 7, 2023 and February 10–23, 2025, each drawing 24,000 consumers. A reader should treat this as predictive and directional audit evidence inside a deployment environment, not a simultaneous controlled causal experiment.

The material states explicitly that the third campaign's checkout increase is deployment evidence for the selected policy, not causal lift against a simultaneous control arm; segment-level conversion lift, opt-out reduction and 60-day retention lift are reserved for deployments with a simultaneous comparison arm or the required telemetry fields. The cost of merging adjacent shopper traits is given only as a posterior-distance prediction of TV=0.051, and a measured cost would still require a controlled re-deployment. Language-model influence is controlled by λT, and the material notes that fit declines at large λT.