Rules made six frontier LLMs ignore the scoreboard; two prompt sentences brought them back

Once a large language model is given an explicit rule, it keeps choosing according to that rule after the win-loss contingencies reverse mid-game, responding weakly to the new outcome feedback. Prior work had reported this rule-based insensitivity in four frontier models; this study extends the test to six models and asks for the first time whether prompting can reduce it. With a correct rule and no induction, the normalized sensitivity index was 0.08 for Grok 4.1 Fast, 0.10 for GPT 5.2 and 0.03 for DeepSeek V3.2, near complete insensitivity, while Gemini 3 Flash, Kimi K2 and Qwen3-235B reached 0.56, 0.43 and 0.40.

Adding two direct tracking sentences to the system prompt, that recent results are more informative than early patterns and that results should guide decisions, markedly restored sensitivity to changing outcomes when a rule was present. Without induction, most models showed a large and persistent drop in optimal selections after the reversal; with induction, that drop was visibly attenuated. The main effect of induction was significant in all six models, the rule-by-induction interaction was significant in all six with effect sizes from 0.09 in Grok 4.1 Fast to 0.28 in GPT 5.2, and the direct tracking induction was effective in every model tested.

The experiment turns rock-paper-scissors into a discrimination-learning task by tying win probabilities to which opponent the model picks rather than to which move it plays. Each trial requires both an opponent choice and a move, but the outcome is determined by the opponent, not the move. The optimal opponent gave an 80% win rate, 10% ties and 10% losses during acquisition, contingencies reversed without warning at trial 41, and the task ran 80 trials with 40 independent sessions per cell, 2,160 sessions across the six models.

The work tests six frontier LLMs on a modified rock-paper-scissors task, with 40 independent sessions per cell and 2,160 sessions in total; the induction is two sentences appended to the system prompt and does not touch model parameters. The authors propose testing these effects on more ecologically valid tasks and pooling human and LLM investigations for direct comparison.

The authors note that all tested models were the most advanced from each provider at the time of data collection, so the results generalize within that population, while smaller or less capable models might be more insensitive and less responsive to tracking inductions. The study did not include models previously identified as particularly sensitive to contingencies, so it cannot show whether induction further helps already flexible models. Metaphorical induction varied by model and fell below no induction in Kimi K2, which the authors attribute tentatively to abstract reasoning demands without a direct test.