Yaroslav Getman

Finnish speech self-supervised pretraining no longer relies only on voice activity detection and language identification to pick data. Those pipelines ensured speech was present, not that it was clear; this pipeline uses ASR confidence as a proxy for pretraining data quality after voice activity detection and deduplication, keeping 471K hours from 2.1M hours of raw broadcast audio. On four benchmarks, confidence filtering gives 3-7% relative WER improvement over language-identification filtering despite a smaller retained training set.

Finnish speech self-supervised pretraining no longer relies only on voice activity detection and language identification to pick data. Those pipelines ensured speech was present, not that it was clear; this pipeline uses ASR confidence as a proxy for pretraining data quality after voice activity detection and deduplication, keeping 471K hours from 2.1M hours of raw broadcast audio. On four benchmarks, confidence filtering gives 3-7% relative WER improvement over language-identification filtering despite a smaller retained training set.