# MoE Load Balancing

In expert-parallel MoE serving, each layer finishes only when its most loaded GPU does, and routing shifts at every decode step. Relief replaces the per-step linear program with an integer bidirectional pairwise relaxation that halves load gaps along replica edges and nears the per-step optimum in a few passes. On decode routing from three open MoE families, scored as effective utilization relative to that optimum, the deployed static scheme reaches 66% to 72%, the GPU kernel 93%, and the on-die engine 99.5%, with the largest recovery over static placement at 37.3 percentage points.

In expert-parallel MoE serving, each layer finishes only when its most loaded GPU does, and routing shifts at every decode step. Relief replaces the per-step linear program with an integer bidirectional pairwise relaxation that halves load gaps along replica edges and nears the per-step optimum in a few passes. On decode routing from three open MoE families, scored as effective utilization relative to that optimum, the deployed static scheme reaches 66% to 72%, the GPU kernel 93%, and the on-die engine 99.5%, with the largest recovery over static placement at 37.3 percentage points.

Data-mixture changes during mixture-of-experts (MoE) training disrupt the per-expert load a router had converged to, yet existing balancing methods were evaluated only on stationary data. This work formulates load balancing as feedback control: each expert's deviation from an equal share is the control error, a per-expert bias added to router scores is the control input, and the auxiliary loss and ALF-LB correspond to a proportional term and a sign-based integral term. Keeping only the integral term, PID (I) attains lower MaxVio than the compared methods before and after a mid-run mixture switch in OLMoE pretraining and in continual pretraining of OLMoE-1B-7B, at comparable perplexity.

Data-mixture changes during mixture-of-experts (MoE) training disrupt the per-expert load a router had converged to, yet existing balancing methods were evaluated only on stationary data. This work formulates load balancing as feedback control: each expert's deviation from an equal share is the control error, a per-expert bias added to router scores is the control input, and the auxiliary loss and ALF-LB correspond to a proportional term and a sign-based integral term. Keeping only the integral term, PID (I) attains lower MaxVio than the compared methods before and after a mid-run mixture switch in OLMoE pretraining and in continual pretraining of OLMoE-1B-7B, at comparable perplexity.