# Post-training

30 related articles

Subscribe to updates

Hugging Face Blog Nov 13, 2026
LLMON gives model context an explicit structure: Granite 4.1 3B and 8B previews cut OPI injection attacks to near zero
IBM researchers released LLMON (LLM Object Notation), a markup language that uses reserved special tokens to mark the boundaries of instructions, data and requirements, together with two experimental preview models trained from Granite 4.1 3B and 8B base checkpoints. The approach turns each semantically coherent component of the context into an addressable block, and an execution block states which instruction to run, which data it operates on, and where the output goes. On the OPI benchmark the LLMON previews avoided 47% more attacks than the best frontier model tested and 95% more than the best open-source model, while general benchmark performance stayed on par with Granite 4.1 Instruct.
IBM researchers released LLMON (LLM Object Notation), a markup language that uses reserved special tokens to mark the boundaries of instructions, data and requirements, together with two experimental preview models trained from Granite 4.1 3B and 8B base checkpoints. The approach turns each semantically coherent component of the context into an addressable block, and an execution block states which instruction to run, which data it operates on, and where the output goes. On the OPI benchmark the LLMON previews avoided 47% more attacks than the best frontier model tested and 95% more than the best open-source model, while general benchmark performance stayed on par with Granite 4.1 Instruct.
LLMON granite-4.1-LLMON-preview Prompt injection defense Post-training
arXiv Nov 7, 2026
Speculative decoding drafters get a new training target: one D-REAL epoch lifts DFlash2 serving throughput 9.4% on GSM8K
Speculative decoding drafters are usually trained with token-level cross-entropy, yet at fixed overhead end-to-end speedup is governed by acceptance length, the number of draft tokens the target model accepts consecutively. D-REAL writes that acceptance length as an expectation along the drafter's own sampled trajectories and builds a differentiable surrogate from acceptance ratios with stop-gradient denominators, giving an unbiased gradient estimator. Across ten benchmarks on the parallel drafters DSpark and DFlash2, one post-training epoch raises macro-average acceptance length from 5.07 to 5.13 and from 4.88 to 5.10; in concurrent GSM8K serving, DFlash2 output throughput rises 9.4%.
Speculative decoding drafters are usually trained with token-level cross-entropy, yet at fixed overhead end-to-end speedup is governed by acceptance length, the number of draft tokens the target model accepts consecutively. D-REAL writes that acceptance length as an expectation along the drafter's own sampled trajectories and builds a differentiable surrogate from acceptance ratios with stop-gradient denominators, giving an unbiased gradient estimator. Across ten benchmarks on the parallel drafters DSpark and DFlash2, one post-training epoch raises macro-average acceptance length from 5.07 to 5.13 and from 4.88 to 5.10; in concurrent GSM8K serving, DFlash2 output throughput rises 9.4%.
Meng Wei（First author）
D-REAL Speculative Decoding Acceptance-Length Optimization Post-training
MiniMax Nov 11, 2026
MiniMax opens H3-Image weights: 4K images in under 10 seconds, up to 5× faster than mainstream tools
MiniMax released H3-Image, an open-weight image generation model built on H3 that supports text-to-image, reference-based generation and image editing, and can produce a 4096×4096 4K image in under 10 seconds, which MiniMax says is up to 5× the speed of mainstream alternatives. Previously, 4K output carried more cost than the common 1K or 2K sizes, which kept it out of everyday creative workflows. The speed comes from changes across the whole post-training pipeline, including data, reinforcement learning, sampling-step reduction, PE and deployment optimization, rather than from a single change.
MiniMax released H3-Image, an open-weight image generation model built on H3 that supports text-to-image, reference-based generation and image editing, and can produce a 4096×4096 4K image in under 10 seconds, which MiniMax says is up to 5× the speed of mainstream alternatives. Previously, 4K output carried more cost than the common 1K or 2K sizes, which kept it out of everyday creative workflows. The speed comes from changes across the whole post-training pipeline, including data, reinforcement learning, sampling-step reduction, PE and deployment optimization, rather than from a single change.
MiniMax
MiniMax H3-Image Text-to-Image Generation Image Editing Post-training
vLLM Blog Nov 6, 2026
VeRL-Omni v0.3.0 brings async execution to multimodal RL training: short answers still favor sync, long ones favor separate-async
VeRL-Omni v0.3.0 extends asynchronous execution to multimodal reinforcement-learning training across text, audio input and diffusion models, and makes the V1 trainer built on TransferQueue and ReplayBuffer the default. Previously a strictly alternating loop left generation and training idle in turn. In MiniCPM-o 4.5 audio-visual question answering, separate-async ran about 9% slower per prompt on natural short answers but about 6.7% faster per prompt at forced 1024-token responses, so the execution mode should follow the workload.
VeRL-Omni v0.3.0 extends asynchronous execution to multimodal reinforcement-learning training across text, audio input and diffusion models, and makes the V1 trainer built on TransferQueue and ReplayBuffer the default. Previously a strictly alternating loop left generation and training idle in turn. In MiniCPM-o 4.5 audio-visual question answering, separate-async ran about 9% slower per prompt on natural short answers but about 6.7% faster per prompt at forced 1024-token responses, so the execution mode should follow the workload.
VeRL-Omni Asynchronous Rollout DiffusionNFT Group Sequence Policy Optimization Post-training
Baseten Oct 24, 2026
Baseten launches an environment network: RL environment building goes to vetted partners, while training and inference stay in-house
Baseten introduced the Baseten Environment Network, handing the construction of reinforcement-learning environments for post-training to a set of vetted partner companies. Previously Baseten's own applied research team built these environments by hand for post-training engagements such as Harvey, Lovable and Gamma, work the company describes as manually intensive and dependent on researchers' judgment. Partners work directly with customers and build environments from the customer's real data, while Baseten handles training and inference deployment; Baseten also plans to open-source a set of frontier-quality environments along with how it audits and trains on them.
Baseten introduced the Baseten Environment Network, handing the construction of reinforcement-learning environments for post-training to a set of vetted partner companies. Previously Baseten's own applied research team built these environments by hand for post-training engagements such as Harvey, Lovable and Gamma, work the company describes as manually intensive and dependent on researchers' judgment. Partners work directly with customers and build environments from the customer's real data, while Baseten handles training and inference deployment; Baseten also plans to open-source a set of frontier-quality environments along with how it audits and trains on them.
Baseten
Baseten Environment Network Post-training RL environments
aclanthology.org Oct 31, 2026
Masked diffusion post-training stops sampling blocks at random: a full-block estimator scores highest on four of five benchmarks
Post-training for masked diffusion language models relies on Monte Carlo estimates of a blockwise surrogate objective, and the existing scheme samples the block index at random, so one block may be evaluated repeatedly while others are never evaluated. A full-block estimator evaluates every block exactly once and averages the results; analysis shows it stays unbiased and removes the between-group variance from block selection. Across five benchmarks in mathematics, planning and coding it reaches the highest accuracy on four, averaging 12.79 points above the random block sampling baseline.
Post-training for masked diffusion language models relies on Monte Carlo estimates of a blockwise surrogate objective, and the existing scheme samples the block index at random, so one block may be evaluated repeatedly while others are never evaluated. A full-block estimator evaluates every block exactly once and averages the results; analysis shows it stays unbiased and removes the between-group variance from block selection. Across five benchmarks in mathematics, planning and coding it reaches the highest accuracy on four, averaging 12.79 points above the random block sampling baseline.
Full-block Estimator Masked Diffusion Models Post-training GRPO
Baseten Oct 30, 2026
Postman hands agent workflows to Baseten inference: enterprise deployment gains a division of labor
Baseten and Postman announced a partnership that splits enterprise AI agent work in two: Postman's Agent Stack prepares APIs, controls which systems agents can reach and governs their actions, while Baseten supplies high-performance inference, multi-cloud GPU capacity and post-training. Baseten says its model APIs placed in the top two providers on Artificial Analysis for throughput and latency across seven 2026 launches, and cites Harvey post-training a 27B open model to match closed-source frontier performance on its legal agent benchmark.
Baseten and Postman announced a partnership that splits enterprise AI agent work in two: Postman's Agent Stack prepares APIs, controls which systems agents can reach and governs their actions, while Baseten supplies high-performance inference, multi-cloud GPU capacity and post-training. Baseten says its model APIs placed in the top two providers on Artificial Analysis for throughput and latency across seven 2026 launches, and cites Harvey post-training a 27B open model to match closed-source frontier performance on its legal agent benchmark.
Baseten
Baseten-Postman partnership Agent serving infrastructure Post-training
Modal Blog Oct 21, 2026
Modal ships Dojo, a post-training library that turns production agent traces into LLM training runs
Modal released Modal Dojo, a post-training library aimed at teams that already run agents in production and collect evaluations and traces. It bundles model selection, data loading, training recipes and metric observability behind one interface; the official example launches training of Qwen3_8_27B on 120 GSM8K training samples. Built on open frameworks such as Miles and Slime, it is said to be validated on models from half a billion to multi-trillion parameters.
Modal released Modal Dojo, a post-training library aimed at teams that already run agents in production and collect evaluations and traces. It bundles model selection, data loading, training recipes and metric observability behind one interface; the official example launches training of Qwen3_8_27B on 120 GSM8K training samples. Built on open frameworks such as Miles and Slime, it is said to be validated on models from half a billion to multi-trillion parameters.
Modal Dojo Post-training Reinforcement Learning
arXiv Oct 28, 2026
An open post-training recipe makes fact-checking models harsher on low-status posters, while seven deployed models show no such tilt
On fact-checking claims with randomized poster profiles, the open Tülu-3 post-training recipe makes OLMo-2 32B and Tülu-3 70B judge claims from lower-status posters false more often, an effect under one percentage point but statistically reliable, exceeding training drift and replicating on independent claim sets; Meta's, Alibaba's and Mistral's recipes do not produce it. In OLMo-2 32B the model does not notice the job more but interprets it differently, with a lower-status job pushing the base model toward true and the post-trained model toward false, a flip carried by a single residual-stream direction at supervised fine-tuning. None of seven deployed commercial models doubts lower-status posters more.
On fact-checking claims with randomized poster profiles, the open Tülu-3 post-training recipe makes OLMo-2 32B and Tülu-3 70B judge claims from lower-status posters false more often, an effect under one percentage point but statistically reliable, exceeding training drift and replicating on independent claim sets; Meta's, Alibaba's and Mistral's recipes do not produce it. In OLMo-2 32B the model does not notice the job more but interprets it differently, with a lower-status job pushing the base model toward true and the post-trained model toward false, a flip carried by a single residual-stream direction at supervised fine-tuning. None of seven deployed commercial models doubts lower-status posters more.
Nouar AlDahoul（First author）
Speaker bias auditing in LLM fact-checking Post-training Supervised Fine-tuning Mechanistic Interpretability OLMo-2
Baseten Oct 30, 2026
A 2B decision model returns one token in 20 ms: joey-1 costs a tenth of mammoth-1 per million tokens
Base Labs released joey-1, a 2B decision model that answers label-style calls such as routing a ticket or flagging a message by returning a single token and a calibrated probability in one forward pass, instead of generating text. At production scale it serves the average 2,000-token request at a P50 end-to-end latency of 20 ms, about 2.5x faster than elephant-1 at 49 ms, and at $0.01 per million tokens it costs a tenth of mammoth-1.
Base Labs released joey-1, a 2B decision model that answers label-style calls such as routing a ticket or flagging a message by returning a single token and a calibrated probability in one forward pass, instead of generating text. At production scale it serves the average 2,000-token request at a P50 end-to-end latency of 20 ms, about 2.5x faster than elephant-1 at 49 ms, and at $0.01 per million tokens it costs a tenth of mammoth-1.
Baseten
joey-1 decision model Qwen3.5 Post-training