# Qwen3

On AMD Instinct MI355X, Qwen3-30B-A3B-Instruct trained with fully asynchronous agentic RL after supervised fine-tuning raised pass@1 from 7.3% to 27.5% on a 100-task stratified subset of SWE-bench Verified, with RL tasks drawn from repositories absent from that benchmark. The bottlenecks were long-tail trajectories and CPU sandbox scheduling rather than GPU compute: capping trajectory length cut step time from 850 s to 284 s, and cloud sandboxes at 384 concurrent trajectories lifted rollout throughput from about 860 to about 3,480 trajectories per hour.

On AMD Instinct MI355X, Qwen3-30B-A3B-Instruct trained with fully asynchronous agentic RL after supervised fine-tuning raised pass@1 from 7.3% to 27.5% on a 100-task stratified subset of SWE-bench Verified, with RL tasks drawn from repositories absent from that benchmark. The bottlenecks were long-tail trajectories and CPU sandbox scheduling rather than GPU compute: capping trajectory length cut step time from 850 s to 284 s, and cloud sandboxes at 384 concurrent trajectories lifted rollout throughput from about 860 to about 3,480 trajectories per hour.

Multi-turn agent distillation now grounds teacher supervision in the interaction state, keeping only executed actions and observations, and the Qwen3-4B student surpasses its 32B teacher on ALFWorld Seen at 43.39%.

Multi-turn agent distillation now grounds teacher supervision in the interaction state, keeping only executed actions and observations, and the Qwen3-4B student surpasses its 32B teacher on ALFWorld Seen at 43.39%.

On 783 de-identified free-text mammography, hip-knee-ankle and brain MR reports, an LLM Council of three general small models plus a chairman reached 83.56% overall accuracy, below the medically trained MedGemma 4B at 88.72%, with the gap driven mainly by hip-knee-ankle classification. A single council member, Qwen3-4B-2507, reached the highest accuracy at 91.69%, above both the council and MedGemma, while the council needed four model calls per report and about 2.44 times MedGemma's peak GPU memory.

On 783 de-identified free-text mammography, hip-knee-ankle and brain MR reports, an LLM Council of three general small models plus a chairman reached 83.56% overall accuracy, below the medically trained MedGemma 4B at 88.72%, with the gap driven mainly by hip-knee-ankle classification. A single council member, Qwen3-4B-2507, reached the highest accuracy at 91.69%, above both the council and MedGemma, while the council needed four model calls per report and about 2.44 times MedGemma's peak GPU memory.

Offline demonstrations, current-policy rollouts, and historical rollouts can now be trained under one regression target, and on four math-reasoning benchmarks the unified configuration improves Qwen3-1.7B benchmark-averaged avg@32 and pass@32 over two-stage SFT plus DAPO by 2.33 and 4.44 percentage points.

Offline demonstrations, current-policy rollouts, and historical rollouts can now be trained under one regression target, and on four math-reasoning benchmarks the unified configuration improves Qwen3-1.7B benchmark-averaged avg@32 and pass@32 over two-stage SFT plus DAPO by 2.33 and 4.44 percentage points.

A home-care scheduling agent no longer asks a small model to handle a narrow task cold. Zero-shot small models failed it, and retraining on every behavior change was too slow, so the behavior was first iterated in prompt space on GPT-5.4-nano, then 11.6k recorded production decisions were LoRA-tuned into Qwen3-8B. That model agrees with recorded decisions 0.913 of the time, above the teacher's 72.7% self-agreement when re-run on its own prompts.

A home-care scheduling agent no longer asks a small model to handle a narrow task cold. Zero-shot small models failed it, and retraining on every behavior change was too slow, so the behavior was first iterated in prompt space on GPT-5.4-nano, then 11.6k recorded production decisions were LoRA-tuned into Qwen3-8B. That model agrees with recorded decisions 0.913 of the time, above the teacher's 72.7% self-agreement when re-run on its own prompts.

Fine-tuning a language model on a new task can erode old abilities, and the prior data kept for protection covers only part of what the model knows. Researchers organize task descriptions and examples into Task Banks, let a frozen pre-update model generate its own replay, and test protection on four held-out tasks excluded from bank construction. Merging two partial banks raises mean held-out retention above either bank alone at all three tested sizes, by up to 4.42 percentage points; at 4B, selection informed by the incoming task beats random selection by 2.75 points, an advantage absent at smaller sizes.

Fine-tuning a language model on a new task can erode old abilities, and the prior data kept for protection covers only part of what the model knows. Researchers organize task descriptions and examples into Task Banks, let a frozen pre-update model generate its own replay, and test protection on four held-out tasks excluded from bank construction. Merging two partial banks raises mean held-out retention above either bank alone at all three tested sizes, by up to 4.42 percentage points; at 4B, selection informed by the incoming task beats random selection by 2.75 points, an advantage absent at smaller sizes.

On factual question answering with Qwen3-14B, keeping the generator fixed and training a separate answer selector works better than training the model itself when to abstain. On TriviaQA the selector raises mean coverage from 20.2% under the original calibrated confidence filter to 30.9%, while pooled error among scored answers falls from 23.0% to 14.5%; on NQ-Open its mean utility still stays below always abstaining, so reliable cross-dataset abstention remains unresolved.

On factual question answering with Qwen3-14B, keeping the generator fixed and training a separate answer selector works better than training the model itself when to abstain. On TriviaQA the selector raises mean coverage from 20.2% under the original calibrated confidence filter to 30.9%, while pooled error among scored answers falls from 23.0% to 14.5%; on NQ-Open its mean utility still stays below always abstaining, so reliable cross-dataset abstention remains unresolved.

In a controlled setting, the authors trained low-rank adapters to make binary decisions for fictitious characters according to latent linear preference functions, found that accurate self-reporting of learned preferences emerges from decision-task fine-tuning alone without explicit self-report supervision, and showed that faithful models have significantly higher attribution similarity between the decision and self-report tasks (mean 0.34 versus 0.08; paired-bootstrap 95% CI [0.16, 0.36]), allowing faithful and unfaithful self-report to be distinguished without understanding the report content.

In a controlled setting, the authors trained low-rank adapters to make binary decisions for fictitious characters according to latent linear preference functions, found that accurate self-reporting of learned preferences emerges from decision-task fine-tuning alone without explicit self-report supervision, and showed that faithful models have significantly higher attribution similarity between the decision and self-report tasks (mean 0.34 versus 0.08; paired-bootstrap 95% CI [0.16, 0.36]), allowing faithful and unfaithful self-report to be distinguished without understanding the report content.