# Bidirectional Preference Synthesis

Kuaishou Technology and Nanjing University introduce Bidirectional Preference Synthesis (BPS): for each validated boundary failure, BPS keeps the conventional forward pair under the original prompt and adds a reverse pair under a synthesized achieved prompt, so the same response is rejected where it is wrong and chosen where it is right; on Qwen3-4B-Instruct-2507, BPS preserves original-side pairwise ranking while raising achieved-side ranking accuracy on held-out crossed anchors from 6.8% to 62.3%, without changing the DPO objective, training a reward model, or requiring online sampling.

Kuaishou Technology and Nanjing University introduce Bidirectional Preference Synthesis (BPS): for each validated boundary failure, BPS keeps the conventional forward pair under the original prompt and adds a reverse pair under a synthesized achieved prompt, so the same response is rejected where it is wrong and chosen where it is right; on Qwen3-4B-Instruct-2507, BPS preserves original-side pairwise ranking while raising achieved-side ranking accuracy on held-out crossed anchors from 6.8% to 62.3%, without changing the DPO objective, training a reward model, or requiring online sampling.