# Max(MI(S;t))

Autoregressive language models normally learn next-token prediction in a fixed left-to-right order; Max(MI(S;t)) instead ranks target tokens by their mutual information with the source and already-selected targets before training, exposing informative targets earlier. On GPT-style models it reaches 94.96% average accuracy across arithmetic tasks, against 74.84% for next-token prediction and 86.29% for reverse order, close to the 97.33% upper bound from exhaustively testing 24 orders; on WikiText-2 it lowers average perplexity from 13.58 to 10.77. Only the training-target organization changes; architecture and autoregressive decoding stay the same.

Autoregressive language models normally learn next-token prediction in a fixed left-to-right order; Max(MI(S;t)) instead ranks target tokens by their mutual information with the source and already-selected targets before training, exposing informative targets earlier. On GPT-style models it reaches 94.96% average accuracy across arithmetic tasks, against 74.84% for next-token prediction and 86.29% for reverse order, close to the 97.33% upper bound from exhaustively testing 24 orders; on WikiText-2 it lowers average perplexity from 13.58 to 10.77. Only the training-target organization changes; architecture and autoregressive decoding stay the same.