Name any loudness in dB SPL: a frozen WavLM converter hits 83% of the target intensity change

The control signal for vocal intensity conversion becomes a continuous sound pressure level in dB, an absolute acoustic quantity measured one meter from the speaker's mouth, instead of a reference utterance or a few discrete categories. Earlier systems either supplied a reference recording that already carried the desired effort or selected among the discrete noise or effort conditions of their corpus. On the calibrated AVID corpus, with one converter architecture and one training objective, the displacement slope on predicted intensity is 0.83 for WavLM layer 6, 0.62 for log-mel, and 0.58 for SpeechTokenizer.

Conversion edits only the frame sequence produced by a frozen encoder, leaving both encoder and decoder untrained, so the output inherits the reconstruction quality of the chosen pair. The prior route retrained the representation itself, for instance distilling the second residual-vector-quantization layer of SpeechTokenizer from a dedicated vocal-effort encoder. The converter is a six-layer bidirectional Transformer trained as a conditional GAN with a least-squares loss, a projection discriminator conditioned on the level, and cycle consistency, using no parallel data and no intensity predictor inside the generator objective.

In listening tests, perceived loudness of the converted speech rises with the target, and listeners scale converted changes much as they scale natural ones. Earlier systems had no target expressed in physical units, so they could not report whether a requested level was reached. Eight evenly spaced targets from 45 to 75 dB SPL over four test speakers gave 64 stimuli, paired with four anchors into 256 pairs plus 32 control pairs; 54 of 81 listeners passed screening, and the WavLM layer-6 conversions show a perceptual slope of 0.69 against 0.68 for original recordings.

The conversion targets calibrated recordings with a known mouth-to-microphone distance, with target values inside the training range, and it preserves the content and speaker identity of the source. Code, model weights, and converted test samples are publicly available. The authors propose that, since no feature space leads on every measure, a hybrid feature set with a dedicated decoder could combine their strengths; more importantly, estimating training source levels with an automatic intensity predictor would allow training on larger, uncalibrated corpora with an independent predictor for evaluation.

The model controls intensity relative to the source recording but is less consistent at reaching an absolute perceived level: for the same target, perceived level varies between source recordings with a standard deviation of about 6 dB. Conversions displace fundamental frequency and spectral measures less than original recordings do, which the authors say leaves room for improvement. Calibrated corpora covering a wide range of vocal intensities are scarce, which limits the training data. The perceptual evaluation covers only the WavLM layer-6 conversions.