RT-SFT turns monolingual corpora into pseudo-parallel data via roundtrip translation and beats few-shot in-context learning across four style domains
Related research and updates
Synopsis
The work introduces RT-SFT: neural MT roundtrip translation through a pivot language serves as a large-scale, task-training-free style-stripping normalizer, converting a monolingual in-style corpus into a pseudo-parallel corpus on which an instruction-tuned LLM is LoRA-finetuned as the stylizer, with the same normalizer applied to test queries to keep the stylizer in-distribution; across four style domains RT-SFT outperforms state-of-the-art methods such as few-shot in-context learning by considerable margins, and the paper also reports effective retrieval augmentation for expert style domains with strict terminology and naming conventions.
Figure 1: Our proposed workflow for finetuning large language models (LLMs) for text style transfer (TST) using only non-parallel data in the target domain. A bilingual general-domain parallel dataset is used to train a pair of neural machine translation (NMT) models capable of translating between English and a pivot language. We then obtain RT-normalized texts of the original in-domain texts by roundtrip translating the in-domain set with the NMT models. This enables supervised finetuning of LLMs for TST, where we finetune LLMs for RT-normalized-domain to target-domain transfer using the synthetic parallel corpus.
arXivInterpretation
Normalization is turned from an inference-time patch into a data-generation tool: roundtrip translation through a pivot language strips stylistic signal while preserving content, yielding a pseudo-parallel corpus from a monolingual in-style corpus. Prior work used a lightweight, task-specific paraphraser applied only at test time, feeding a correspondingly small stylizer; here a neural MT system already trained on hundreds of millions of general-domain sentence pairs serves as an off-the-shelf, large-scale style-stripping normalizer without any task-specific training. Based on the observation and pipeline stated in the abstract: neural MT trained on hundreds of millions of general-domain sentence pairs preserves content while regressing toward generic phrasing, and roundtrip translation produces the pseudo-parallel corpus. The visible text gives no corpus sizes, language pairs, or human-evaluation details.
RT-SFT outperforms state-of-the-art methods such as few-shot in-context learning by considerable margins across four style domains. Relative to training-free approaches like few-shot in-context learning, RT-SFT LoRA-finetunes an instruction-tuned LLM as the stylizer and keeps test queries in-distribution by applying the same normalizer. The abstract reports results 'across four style domains' and 'by considerable margins', but the visible text provides no specific metrics, baseline list, or statistical tests.
For expert style domains with strict terminology and naming conventions, effective retrieval augmentation methods are reported. Retrieval augmentation is brought into style transfer to handle terminology and naming conventions that general style-transfer methods do not specifically address. The abstract only states that the work reports on 'effective retrieval augmentation methods'; retrieval sources, granularity, and ablations are not given.
Perspective
The result targets settings where a target-style rewrite is needed and only a monolingual in-style corpus is available, for languages and domains where reliable pivot-language roundtrip translation exists; for expert style domains with strict terminology and naming conventions, the method includes retrieval augmentation as a companion component. It positions normalization as a training-data generation step, so its benefit depends on whether the MT system used indeed preserves content while regressing toward generic phrasing.
The visible text is abstract-level, lacking specific evaluation metrics, baseline lists, the names of the four style domains, corpus and language-pair sizes, human evaluation, and statistical significance, so the magnitude of the 'considerable margins' cannot be judged. Retrieval sources, granularity, and ablations are also not described in the visible text. The trade-off between content preservation and style stripping in roundtrip translation, and the effect of pivot-language choice, remain open questions to watch.