Geo-distributed LLM serving stops reading only grid carbon: counting batching and queueing cuts emissions up to 60% at about 3% SLO violation
When geo-distributed LLM serving schedulers account for an engine's batching and queueing state, they cut carbon emissions by up to 60% relative to round-robin under MLPerf per-token latency targets, with an average SLO violation of about 3%.