# RubatoCodec

Variable-frame-rate speech codecs allocate tokens by content, but threshold-based allocation lets content set the token count, so at a fixed setting the sequence length a speech generator must produce varies across utterances and datasets. RubatoCodec adds an exact utterance-level event budget and streaming decoding to X-Codec-2, using changes in codec features and waveform spectra to cut speech into variable-duration events, each represented by discrete codes plus an explicit duration. Controlled experiments show adaptive allocation beats fixed-rate coding in reconstruction fidelity and intelligibility at low event rates; in a matched text-to-speech comparison the representation yields lower WER and higher UTMOS and UTMOSv2 with comparable speaker similarity.

Variable-frame-rate speech codecs allocate tokens by content, but threshold-based allocation lets content set the token count, so at a fixed setting the sequence length a speech generator must produce varies across utterances and datasets. RubatoCodec adds an exact utterance-level event budget and streaming decoding to X-Codec-2, using changes in codec features and waveform spectra to cut speech into variable-duration events, each represented by discrete codes plus an explicit duration. Controlled experiments show adaptive allocation beats fixed-rate coding in reconstruction fidelity and intelligibility at low event rates; in a matched text-to-speech comparison the representation yields lower WER and higher UTMOS and UTMOSv2 with comparable speaker similarity.