Mid-training a language model on raw video lifts video benchmarks by 2.9 points and image benchmarks by 5.1 points while preserving text ability
This work asks whether raw web video, with no captions and no text loss, can serve as mid-training data for a pretrained language model: frames are encoded into continuous visual tokens and the model learns to predict the next visual token, Qwen3-1.7B is mid-trained on raw clips from YT-Temporal-1B and then given the same image-text instruction tuning as a model without mid-training, yielding 2.9 points higher on video benchmarks and 5.1 points higher on image benchmarks, with text average rising from 48.0 to 48.9, and caption prediction offering no advantage over visual-token prediction.