# Asynchronous RL

Reinforcement-learning training of reasoning language models chains generation, assessment and training across several large models each step, and existing systems usually optimize only part of that design space. RPEG places pipeline structure, including bounded asynchrony and actor–critic sharing, and execution choices such as stage fusion, fine-grained scheduling and multi-dimensional parallelism into one graph searched jointly, backed by a hardware-calibrated analytical performance model. On up to 256 GH200 GPUs its runtime is up to 2x faster end to end than ReaL, cuts step latency by up to 2.48x versus verl, and raises hardware utilization by up to 30%.

Reinforcement-learning training of reasoning language models chains generation, assessment and training across several large models each step, and existing systems usually optimize only part of that design space. RPEG places pipeline structure, including bounded asynchrony and actor–critic sharing, and execution choices such as stage fusion, fine-grained scheduling and multi-dimensional parallelism into one graph searched jointly, backed by a hardware-calibrated analytical performance model. On up to 256 GH200 GPUs its runtime is up to 2x faster end to end than ReaL, cuts step latency by up to 2.48x versus verl, and raises hardware utilization by up to 30%.

Foundation models are usually trained in stages, with both the data and the learning algorithm changing at each one. Unified training writes pretraining, midtraining, supervised fine-tuning and reinforcement learning as maximum-likelihood fitting to target distributions, so static corpora and RL generations mix inside a single trainer, and staged training becomes one special case. In controlled reasoning tasks and a real-world midtraining experiment, it improved the accuracy-compute tradeoff and task coverage while preserving broader capabilities.

Foundation models are usually trained in stages, with both the data and the learning algorithm changing at each one. Unified training writes pretraining, midtraining, supervised fine-tuning and reinforcement learning as maximum-likelihood fitting to target distributions, so static corpora and RL generations mix inside a single trainer, and staged training becomes one special case. In controlled reasoning tasks and a real-world midtraining experiment, it improved the accuracy-compute tradeoff and task coverage while preserving broader capabilities.

MintAct introduces a family of vision-language models at 2B, 4B, and 8B scales that unify UI grounding, multi-step navigation across mobile, desktop, and web, and visual tool use in a single set of weights through a shared screenshot-and-pixel observation space, prompt-conditioned per-domain action sets, and balanced cross-domain mixing, backed by asynchronous RL infrastructure hosting hundreds of concurrent instances, reaching 48.9 on OSWorld-Verified, 39.1 on Online-Mind2Web, and 67.0 on AndroidWorld while matching or exceeding size-matched per-domain specialists.

MintAct introduces a family of vision-language models at 2B, 4B, and 8B scales that unify UI grounding, multi-step navigation across mobile, desktop, and web, and visual tool use in a single set of weights through a shared screenshot-and-pixel observation space, prompt-conditioned per-domain action sets, and balanced cross-domain mixing, backed by asynchronous RL infrastructure hosting hundreds of concurrent instances, reaching 48.9 on OSWorld-Verified, 39.1 on Online-Mind2Web, and 67.0 on AndroidWorld while matching or exceeding size-matched per-domain specialists.