# Mixture of Hidden Markov Trees

Transformer-based masked diffusion models now have an end-to-end statistical guarantee: when the data distribution is well approximated by a mixture of hidden Markov trees, the training samples needed for a target KL accuracy are governed by latent structural complexity rather than growing exponentially with the sequence space. The analysis covers both the parallelization error of parallel unmasking and the finite-sample estimation error of the mask predictor, and the transformer architecture and sampling procedure need no knowledge of tree topologies or component parameters, adapting to unknown latent structure.

Transformer-based masked diffusion models now have an end-to-end statistical guarantee: when the data distribution is well approximated by a mixture of hidden Markov trees, the training samples needed for a target KL accuracy are governed by latent structural complexity rather than growing exponentially with the sequence space. The analysis covers both the parallelization error of parallel unmasking and the finite-sample estimation error of the mask predictor, and the transformer architecture and sampling procedure need no knowledge of tree topologies or component parameters, adapting to unknown latent structure.