# Visual Tokenizer

Visual generation is split into two stages: autoregressive pretraining acquires visual knowledge, and diffusion post-training renders it into pixels. In controlled comparisons at roughly 400M parameters, the autoregressive-then-diffusion route reached 65.5 on GenEval, above 61.2 for pixel diffusion trained from scratch, and a semantic tokenizer needed neither an image decoder nor a reconstruction loss to support rendering. Allocating compute by fitted scaling laws, a roughly 3B-parameter text-to-image model trained in about 6,600 H100 GPU-hours scored 0.86 on GenEval and 87.78 on DPG-Bench.

Visual generation is split into two stages: autoregressive pretraining acquires visual knowledge, and diffusion post-training renders it into pixels. In controlled comparisons at roughly 400M parameters, the autoregressive-then-diffusion route reached 65.5 on GenEval, above 61.2 for pixel diffusion trained from scratch, and a semantic tokenizer needed neither an image decoder nor a reconstruction loss to support rendering. Allocating compute by fitted scaling laws, a roughly 3B-parameter text-to-image model trained in about 6,600 H100 GPU-hours scored 0.86 on GenEval and 87.78 on DPG-Bench.

Peking University and Singapore University of Technology and Design propose DC-SAE, a decoupled compact semantic autoencoder that pairs a frozen semantic encoder for a generation-friendly compact latent with a trainable pixel encoder for low-level detail, achieving 29.79 PSNR and 3.37 gFID on ImageNet at 32x spatial compression, improving over DC-AE by 13.5% in PSNR and 54.9% in gFID with a 4.41x speedup, while a B-parameter DiT reaches 0.84 on GenEval and 86.007 on DPG-Bench.

Peking University and Singapore University of Technology and Design propose DC-SAE, a decoupled compact semantic autoencoder that pairs a frozen semantic encoder for a generation-friendly compact latent with a trainable pixel encoder for low-level detail, achieving 29.79 PSNR and 3.37 gFID on ImageNet at 32x spatial compression, improving over DC-AE by 13.5% in PSNR and 54.9% in gFID with a 4.41x speedup, while a B-parameter DiT reaches 0.84 on GenEval and 86.007 on DPG-Bench.