# ROSETok

Speech tokenizers compress continuous speech into discrete tokens and serve as the front end of ASR systems and large audio-language models. T-SemAttack skips the full recognition pipeline and instead jointly drifts the representation spaces of several semantic encoders, with a psychoacoustic masking constraint keeping the noise acceptable to listeners while pulling downstream semantic encoding off course. On bilingual multi-speaker data it raises word error rates on nine open-source and black-box ASR systems from about 1–4% on clean speech to 20.004–63.555%, and it introduces ROSETok, a tokenizer trained with noisy training and semantic distillation.

Speech tokenizers compress continuous speech into discrete tokens and serve as the front end of ASR systems and large audio-language models. T-SemAttack skips the full recognition pipeline and instead jointly drifts the representation spaces of several semantic encoders, with a psychoacoustic masking constraint keeping the noise acceptable to listeners while pulling downstream semantic encoding off course. On bilingual multi-speaker data it raises word error rates on nine open-source and black-box ASR systems from about 1–4% on clean speech to 20.004–63.555%, and it introduces ROSETok, a tokenizer trained with noisy training and semantic distillation.