Jingyuan Li

In expert-parallel MoE serving, each layer finishes only when its most loaded GPU does, and routing shifts at every decode step. Relief replaces the per-step linear program with an integer bidirectional pairwise relaxation that halves load gaps along replica edges and nears the per-step optimum in a few passes. On decode routing from three open MoE families, scored as effective utilization relative to that optimum, the deployed static scheme reaches 66% to 72%, the GPU kernel 93%, and the on-die engine 99.5%, with the largest recovery over static placement at 37.3 percentage points.

In expert-parallel MoE serving, each layer finishes only when its most loaded GPU does, and routing shifts at every decode step. Relief replaces the per-step linear program with an integer bidirectional pairwise relaxation that halves load gaps along replica edges and nears the per-step optimum in a few passes. On decode routing from three open MoE families, scored as effective utilization relative to that optimum, the deployed static scheme reaches 66% to 72%, the GPU kernel 93%, and the on-die engine 99.5%, with the largest recovery over static placement at 37.3 percentage points.