Chess engines can outplay anyone but cannot say why: a training-free method lets language models find reusable motifs and nearly triples explanation scores
Chess engines play far beyond human ability yet cannot say why they prefer a move. A study first builds an engine-grounded score for explanations and finds that eight language models prompted directly lean on surface board features; it then introduces BinProp, a training-free method in which models iteratively propose and filter binary chess motifs on winning and losing positions, then use them to explain specific moves. On chess, the eight-model average rises to nearly three times the direct-prompting baseline, and the same setup extends to Go, draughts and shogi.