Convolutional VocalTrax drops the sample-by-sample recursion: tract passes run 47 to 63 times faster on an A100, and reconstructions are recognized more often
Synopsis
Convolutional VocalTrax replaces the sample-by-sample tract recursion in differentiable articulatory synthesis with fixed-block convolution: it computes the exact frequency response of the same waveguide and applies it as a causal convolution in 5.8 ms blocks, so gradients pass through FFTs and a 43-step recursion along the tract. On 600 AudioMNIST recordings, the tract forward and backward pass runs 47 to 63 times faster on an A100, and a wav2vec 2.0 classifier trained only on real speech recognizes its reconstructions 14.5% of the time versus 10.0% for VocalTrax, although the log-mel distance is 11% higher.
Interpretation
Fitting an articulatory synthesizer no longer requires backpropagating through time. Convolutional VocalTrax sends gradients through frequency-domain transforms and a short recursion along the tract instead of through a waveguide updated twice per audio sample. The VocalTrax tract is a Kelly-Lochbaum waveguide advanced twice per audio sample, about 88,000 sequential updates per second of audio, and backpropagation crosses that whole recurrence. On an A100, the tract forward plus backward pass is 47 to 63 times faster for 1 s to 60 s inputs, with both costs growing linearly with duration; complete 1000-step fits of three recordings fall from 12 to 18 minutes to 3.8 to 5.8 minutes.
The method treats the tract as stationary within short windows and convolves with its frequency response block by block. This rests on two assumptions common in speech acoustics: tract geometry changes little within a few milliseconds, and the lossy tract's impulse response decays within tens of milliseconds. The implementation uses 5.8 ms blocks and truncates the impulse response at 93 ms, obtaining the filter by inverse FFT on 8192 frequencies; for fixed geometry the convolution matches VocalTrax output to machine-precision relative error in double precision.
Reconstructed speech becomes more recognizable even though the spectral objective worsens. A frozen wav2vec 2.0 with a linear head, trained only on real speech, recognizes VocalTrax reconstructions at 10.0%, near chance. The same classifier recognizes Convolutional VocalTrax reconstructions at 14.5%, a 4.5-point gain with a paired bootstrap 95% interval of 2.2 to 6.6, while log-mel error is 11.3% higher.
Perspective
The demonstrated scope is 600 AudioMNIST recordings at 44.1 kHz with a 44-segment tract and 1000-step AdamW fits, with timings on one A100 at batch size one in single precision. The authors propose precomputing the source phase for fixed pitch to bring end-to-end speed-up close to the estimated 17 to 25 times, and evaluating longer utterances with listening tests.
The authors note that absolute recognition rates for synthetic speech are low and that predictions for all synthetic conditions concentrate on a few digits, so these numbers indicate relative rather than practical intelligibility. The original VocalTrax study's 73.5% for real recordings and 20.0% for reconstructions could not be reproduced, though all conditions are scored by the same classifiers. Unvoiced frames use a 1 Hz placeholder, putting about 45% of output energy below 60 Hz against 3.8% in the recordings.