Chenyang Le

CRESiST replaces prefix-incremental generation in LLM-based simultaneous speech translation with an interleaved sequence of speech chunks and target tokens, so the KV cache is reused across the whole stream and no voice activity detection is needed to cut sentences. An RNN-T Actor samples read-or-write trajectories, a Qwen2.5-omni-7B Critic with Router and Refiner scores them on semantic consistency and latency, and reverse distillation feeds the decisions back to the Actor. On MuST-C and ACL 60/60, unsegmented streaming lands only 1–2 COMET points below the oracle-segmented setting.

CRESiST replaces prefix-incremental generation in LLM-based simultaneous speech translation with an interleaved sequence of speech chunks and target tokens, so the KV cache is reused across the whole stream and no voice activity detection is needed to cut sentences. An RNN-T Actor samples read-or-write trajectories, a Qwen2.5-omni-7B Critic with Router and Refiner scores them on semantic consistency and latency, and reverse distillation feeds the decisions back to the Actor. On MuST-C and ACL 60/60, unsegmented streaming lands only 1–2 COMET points below the oracle-segmented setting.