# RLM Pipeline Execution Graph (RPEG)

Reinforcement-learning training of reasoning language models chains generation, assessment and training across several large models each step, and existing systems usually optimize only part of that design space. RPEG places pipeline structure, including bounded asynchrony and actor–critic sharing, and execution choices such as stage fusion, fine-grained scheduling and multi-dimensional parallelism into one graph searched jointly, backed by a hardware-calibrated analytical performance model. On up to 256 GH200 GPUs its runtime is up to 2x faster end to end than ReaL, cuts step latency by up to 2.48x versus verl, and raises hardware utilization by up to 30%.

Reinforcement-learning training of reasoning language models chains generation, assessment and training across several large models each step, and existing systems usually optimize only part of that design space. RPEG places pipeline structure, including bounded asynchrony and actor–critic sharing, and execution choices such as stage fusion, fine-grained scheduling and multi-dimensional parallelism into one graph searched jointly, backed by a hardware-calibrated analytical performance model. On up to 256 GH200 GPUs its runtime is up to 2x faster end to end than ReaL, cuts step latency by up to 2.48x versus verl, and raises hardware utilization by up to 30%.