# Model merging

Model merging usually trains every expert at the same LoRA rank, yet each domain has its own best rank. Across 36 experts on mathematics, instruction following and multilingual data at four ranks and three model scales, six merge methods were compared at uniform ranks and at each domain's own best rank: simple averaging and loss-utility component selection are unaffected and gain up to 1.8 pp of multi-domain accuracy, while magnitude-based conflict-resolution methods such as TIES and KnOTS are inconsistent and can lose up to 4 pp. The cause is that selecting components by size discards the instruction adapter, whose singular values are smallest, almost entirely, and IFEval falls by 16 to 25 pp.

Model merging usually trains every expert at the same LoRA rank, yet each domain has its own best rank. Across 36 experts on mathematics, instruction following and multilingual data at four ranks and three model scales, six merge methods were compared at uniform ranks and at each domain's own best rank: simple averaging and loss-utility component selection are unaffected and gain up to 1.8 pp of multi-domain accuracy, while magnitude-based conflict-resolution methods such as TIES and KnOTS are inconsistent and can lose up to 4 pp. The cause is that selecting components by size discards the instruction adapter, whose singular values are smallest, almost entirely, and IFEval falls by 16 to 25 pp.

Isolating the reinforcement-learning-stage parameter update of a language model, keeping only the single dominant direction of each matrix rescaled to the original magnitude, and injecting it into the language modules of a vision-language model improves matched full-update interpolation in 12 of 15 backbone-benchmark comparisons across three model families and five visual-reasoning benchmarks, including an 8.55 percentage-point MathVision gain on the Qwen recipient. Controls show that matching update magnitude or using arbitrary low-rank directions does not reproduce these gains, indicating that what is acquired during post-training and what remains transferable across models are not the same thing.

Isolating the reinforcement-learning-stage parameter update of a language model, keeping only the single dominant direction of each matrix rescaled to the original magnitude, and injecting it into the language modules of a vision-language model improves matched full-update interpolation in 12 of 15 backbone-benchmark comparisons across three model families and five visual-reasoning benchmarks, including an 8.55 percentage-point MathVision gain on the Qwen recipient. Controls show that matching update magnitude or using arbitrary low-rank directions does not reproduce these gains, indicating that what is acquired during post-training and what remains transferable across models are not the same thing.