# Autoregressive Token Generation

The limits of token-by-token generation are now fully characterized: a sampler with little private state cannot sample a pseudorandom generator's output, while an insertional sampler that may write at any position can.

The limits of token-by-token generation are now fully characterized: a sampler with little private state cannot sample a pseudorandom generator's output, while an insertional sampler that may write at any position can.

The authors propose MPAD: a 3D autoencoder compresses MRI volumes into an isotropic latent representation, a 2D diffusion model reconstructs masked target-contrast latent slices conditioned on source-contrast slices, unmasked target slices, and a text modality prompt, and inference generates slices autoregressively per plane while propagating completed volumes as priors to orthogonal planes, achieving full-volume 3D synthesis with 2D operations, reducing training and inference FLOPs by 7x and 3x versus 3D latent diffusion baselines, and improving PSNR, SSIM, and NMSE on ADNI and IXI.

The authors propose MPAD: a 3D autoencoder compresses MRI volumes into an isotropic latent representation, a 2D diffusion model reconstructs masked target-contrast latent slices conditioned on source-contrast slices, unmasked target slices, and a text modality prompt, and inference generates slices autoregressively per plane while propagating completed volumes as priors to orthogonal planes, achieving full-volume 3D synthesis with 2D operations, reducing training and inference FLOPs by 7x and 3x versus 3D latent diffusion baselines, and improving PSNR, SSIM, and NMSE on ADNI and IXI.

This work introduces two learned discrete latent protein languages: PLL, a 4,096-state contextual alphabet built on a frozen ESM-2 encoder on the sequence side, and SLL, which adapts GCP-VQVAE Lite with auxiliary sequence and confidence supervision while remaining decodable to backbone coordinates; under matched downstream training, PLLM has a fitted compute-scaling exponent of 0.038 versus 0.020 for the amino acid autoregressive model and reduces the fraction of samples below a 1.5-bit residue-composition entropy threshold by 54%, while replacing the original tokenizer with SLL reduces best validation perplexity by 34% for sequence-to-structure prediction.

This work introduces two learned discrete latent protein languages: PLL, a 4,096-state contextual alphabet built on a frozen ESM-2 encoder on the sequence side, and SLL, which adapts GCP-VQVAE Lite with auxiliary sequence and confidence supervision while remaining decodable to backbone coordinates; under matched downstream training, PLLM has a fitted compute-scaling exponent of 0.038 versus 0.020 for the amino acid autoregressive model and reduces the fraction of samples below a 1.5-bit residue-composition entropy threshold by 54%, while replacing the original tokenizer with SLL reduces best validation perplexity by 34% for sequence-to-structure prediction.