WaveGen folds zero-shot TTS into one model: text goes straight to waveform, hitting WER 1.47 and SIM 0.75 on Seed-en

Zero-shot speech synthesis can drop every separately trained intermediate representation: one model emits 24 kHz raw waveform from text, with no neural audio codec, Mel vocoder, VAE latent, duration predictor or self-supervised semantic representation in the generation path. Zero-shot TTS had settled into multi-stage pipelines: codec models depend on the capacity of a neural audio codec, Mel models need a separately trained vocoder, latent diffusion models need an extra pretrained latent, and earlier end-to-end models suffered from limited scalability, training instability and weaker zero-shot quality. On the Seed-en benchmark, the 0.6B WaveGen reaches WER 1.47 and SIM 0.75, while the same comparison table lists 0.6B Qwen3-TTS at WER 1.68 and SIM 0.70 and 1.7B Qwen3-TTS at WER 1.50 and SIM 0.71; WaveGen uses open Emilia data, one-stage training and no REPA.

The mechanism generates the waveform trajectory together with a low-resolution spectral trajectory inside one model, where the spectrum serves only as internal guidance rather than an external intermediate output. Applying conditional flow matching directly to raw waveform makes text-speech alignment hard at high temporal resolution, and earlier waveform-level diffusion work still relied on Mel-spectrogram conditioning or pretrained BERT text representations. Without Spectral Forcing the model fails to align, at WER 123.67 and SIM 0.19; with a 64-bin Mel internal spectral trajectory WER falls to 4.96 and SIM reaches 0.47, while 128 bins give WER 18.67 and 32 bins give WER 4.26, so 64 bins were chosen as the trade-off among alignment, convergence and high-frequency detail.

The demonstrated scope is zero-shot TTS on 24 kHz speech with open Emilia data, evaluated mainly by WER and SIM on Seed-en, speaker similarity on Seed-zh, and MTurk naturalness and similarity ratings; the 0.6B model was trained for 11 epochs on eight GPUs and the other models for four epochs.

Limited compute restricted large-scale training to four epochs, and the authors expect longer training to help further; REPA was tested only in a small-scale preliminary experiment on WaveGen-S with LibriTTS and is not used in the main models; MP3-compressed Emilia audio makes high-frequency components above 8 kHz hard to generate; the authors plan to extend the model to 48 kHz waveform generation.