Microsoft scales its inference silicon to multi-gigawatt: Maia 300 promises 40% more tokens per megawatt, production in 2027

Microsoft is moving its in-house inference accelerator from a single generation to a multi-generation, multi-gigawatt infrastructure line: Maia 200 is in production and Maia 300 is set to begin production deployments in 2027. Azure Maia had been one element of Microsoft's AI infrastructure strategy; it is now positioned as the primary inference system for Microsoft products and services and explicitly planned at multi-gigawatt scale. Maia 200 is already serving Microsoft production workloads including Microsoft Copilot, running OpenAI models, Microsoft's own MAI models and leading open-source model families, with GitHub Copilot and Microsoft Foundry to follow by end of year.

Maia 300 is designed for long-context reasoning, latency-sensitive conversational experiences and sustained multi-step agentic tasks, using an inference-first architecture that balances compute, memory bandwidth, networking and power efficiency at cloud scale. Against Maia 200, a full 96-accelerator Maia 300 configuration delivers up to 7x the rack-scale token throughput. Microsoft expects Maia 300 to deliver 40% more tokens per megawatt than leading GPU-based systems; Maia 200 delivers up to 1.7x more tokens per watt for built-in AI capabilities in Teams, Copilot and Cowork compared with leading GPU infrastructure, and is deploying across the United States, Europe and Australia.

Maia 300 treats data movement as a first-class design principle, pairing a software-defined dataflow architecture with AI-assisted kernel optimization to reduce data movement and improve utilization across compute and memory. The bottleneck in large inference systems is shifting from raw compute power to moving data efficiently, so Maia 300 is not designed around compute alone. The architecture spans compute, memory and network resources as a single system rather than moving data through separate communication layers.

A Maia 300 rack system brings together 96 accelerators within a 167-kilowatt power envelope, delivering 27.6 TB of HBM capacity and 1.7 PB/s of aggregate HBM bandwidth, so each rack can serve more simultaneous AI interactions and larger workloads. Eight HBM4 stacks provide 18.0 TB/s of peak memory bandwidth within a 950-watt TDP, and a multi-chiplet architecture separates compute, memory-controller and I/O functions, with compute chiplets built on TSMC's N2P process node. The design supports advanced low-precision formats including enhanced MXFP4 to cut memory and bandwidth requirements for computation and KV-cache storage, enlarging context windows within the same power envelope.

This report is for readers tracking AI infrastructure and inference hardware, and covers Microsoft's announced multi-generation Azure Maia roadmap, the production deployment scope of Maia 200, and the architecture and rack specifications of Maia 300. Microsoft says Maia 300 will begin production deployments in 2027 and plans to share more strategy and technology information at Microsoft Ignite 2026 next month.

Microsoft frames the 40% gain in tokens per megawatt and the 7x rack-scale throughput as expectations for Maia 300 rather than measured results. Production deployment of Maia 300 is set for 2027, and the currently verifiable operating data comes from Maia 200. Microsoft also says a technical paper on Maia 300 architectural details is forthcoming, indicating that some design details are not expanded in this announcement.