PLaMo 3 Translate releases four weight sizes: the 0.8B model beats translation-specific models more than 30 times larger on Japanese-English and Japanese-multilingual pairs

PLaMo 3 Translate releases weights at four sizes from 0.8B to 31B, and the smallest beats existing translation-specific models more than 30 times larger on Japanese-English and Japanese-multilingual translation. Earlier translation-specific models needed far more parameters for comparable translation quality, so small models could not carry high-quality translation on their own. Japanese-English performance averages PFTB, SaaS and WMT24++ COMET-22; multilingual performance averages FLORES+ chrF++ over Japanese-multilingual and English-multilingual, compared against existing translation-specific models including TranslateGemma and Hy-MT2; on English-multilingual the 2B model surpasses all existing translation-specific models.

Training starts from the PLaMo third-generation base, applies two stages of supervised fine-tuning, then direct preference optimization, with 31B-teacher distillation inserted between the two for models of 8B and below. PLaMo 2 Translate gained most of its improvement from the later DPO stage, whereas this generation achieved large gains in supervised fine-tuning alone through data refinement, with training scheme and hyperparameters barely changed. After supervised fine-tuning the 0.8B model already exceeds the 8B PLaMo 2 Translate on Japanese-English; DPO moves benchmark scores only slightly, and under LLM-judge pairwise comparison the 31B model's first DPO round wins 57% and 59% against the supervised fine-tuned version on PFTB English-to-Japanese and Japanese-to-English.

Translations are synthesized not by a single large-model call but by tournament-style data synthesis, back-translation and iterative self-distillation that repeatedly purify training pairs. A single direct translation from Gemma 4 31B IT often leaves room for improvement, while training only on forward synthetic translations degrades the fluency of model output. Tournament synthesis generates several candidates, has Gemma 4 compare them pairwise, merges the loser's strengths into the winner and repeats over rounds; back-translation uses the source and synthetic translation together to revise the source, yielding synthetic-translation-to-revised-source pairs; iterative self-distillation compares new translations against existing data in a final and keeps the better one, applied only to phase-two data.

Distillation builds on DistiLLM-2 and adjusts the learning target with AMiD's mixture distribution, easing the loss blow-up when a student assigns probability to tokens the teacher deems nearly absent. Plain Reverse KL becomes unstable when teacher and student distributions diverge sharply, while the mixture keeps high-probability teacher tokens nearly intact and lets low-probability tokens absorb a little student mass. Forward KL against the teacher is computed on reference-translation sequences and Reverse KL on student-generated sequences, with student sequences regenerated every few dozen steps; distillation lifts small models most, and models of 8B and below then receive iterative DPO as well.

The release covers model weights and quantization formats at four sizes, and the performance conclusions are limited to Japanese-English, Japanese-multilingual and English-multilingual directions on the PFTB, SaaS, WMT24++ and FLORES+ benchmarks; under the license, non-commercial use has no revenue limit, while commercial use is free after registration when revenue or sales are within 1 billion yen.

DPO's improvement is too small to measure on benchmark scores and shows only win rates just above 50% under LLM-judge pairwise comparison, and not every expression improves; performance still trends down as parameter count shrinks, though Japanese-English holds up comparatively well.