Microsoft unveils Azure Maia 300 on October 12, 2026: 96-chip racks

Microsoft unveiled Azure Maia 300, the third generation of its in-house AI accelerator, on October 12, 2026. In a post by Scott Guthrie, the company said Maia 300 will begin production deployments in 2027 and that it will scale the Maia family to multi-gigawatt deployments from that year, making Maia “the primary inference system for Microsoft products and services.”

The chip is built for inference rather than training. Microsoft describes it as designed for long-context reasoning, latency-sensitive conversation and multi-step agentic tasks, and gives two headline claims: it expects Maia 300 to deliver 40% more tokens per megawatt than leading GPU-based systems, and a full 96-accelerator rack delivers up to seven times the rack-scale token throughput of Maia 200. Both are Microsoft’s own figures; the company has not published an independent benchmark.

The chip: eight HBM4 stacks, compute on TSMC N2P

Maia 300 splits functions across chiplets. According to the architecture deep dive published the same day, compute dies are made on TSMC’s N2P process, while memory-controller and I/O functions move to separate dies chosen for bandwidth, connectivity and cost. Microsoft says this raises the system’s compute FLOPs by 36% and its scale-up bandwidth by 14%.

Against Maia 200, Microsoft lists 2.5 times the memory bandwidth, 33% more memory capacity and roughly twice the HBM bandwidth per watt. Memory also changes shape: Maia 200 used a uniform memory organization, while Maia 300 is non-uniform across chiplets, so software can keep data next to the compute region that uses it. The narrower number formats serve the same goal, cutting the bytes moved per token, including for KV cache storage, which Microsoft says allows longer contexts and more concurrent sessions in the same power envelope. Why bandwidth now gets as much attention as compute is covered in our HBM memory explainer.

The rack: 96 accelerators in 167 kW

The Maia 300 rack holds 96 accelerators in 24 trays of four, each tray with its own host processor, memory, storage and cooling, inside a 167-kilowatt envelope. That doubles the accelerator count of a Maia 200 rack. Across the rack Microsoft counts 27.6 TB of HBM and 1.7 PB/s of aggregate HBM bandwidth, which matches 96 times the per-chip figures.

Six high-radix switch ASICs sit in three liquid-cooled switch trays. Power arrives on dual feeds through a 50 V busbar, and liquid cooling removes more than 80% of the rack’s heat; the rest is air-cooled. Microsoft says the rack can connect directly to facility water or reject heat through a heat-exchanger unit in air-cooled data halls, which widens the set of buildings it can go into.

One Ethernet fabric up to 12,288 accelerators

Maia 300 uses a custom AI transport over Ethernet in two tiers. Inside a rack, electrical links connect all 96 accelerators as one pod, with pairs of accelerators in a tray also wired directly. Optical second-tier switches join up to 128 racks into a superpod of 12,288 accelerators.

Microsoft’s argument is that the same communication model covers the whole superpod, with no separate scale-out network card in the path. An accelerator is one switch hop from any peer in its rack and three hops from any peer elsewhere in the superpod. The company says this lets prefill and decode workers be placed separately and move KV state directly between them, and lets a scheduler swap in spare trays from another rack when hardware fails. Microsoft tied the launch to the OCP Global Summit 2026, which opened the same day, where it also contributed power and cooling specifications.

Maia 200 is already in production

The announcement also updates Maia 200. Microsoft says it now serves production workloads including Microsoft Copilot and supports the MAI team’s model development, with GitHub Copilot and Microsoft Foundry to follow by the end of the year. It runs OpenAI models, Microsoft’s MAI models and open-source model families, and is deploying in the United States, Europe and Australia. For the AI features built into Teams, Copilot and Cowork, Microsoft reports up to 1.7 times more tokens per watt than leading GPU infrastructure.

Maia joins the other hyperscaler chips built mainly for their owners’ own services, such as Google’s TPU and AWS Trainium. Microsoft has not said whether Maia 300 will be offered to Azure customers as a rentable instance, and it gives no figure for how many Maia accelerators it plans to deploy beyond “multi-gigawatt.” The company says it will share more at Microsoft Ignite, November 17–20, 2026. Published specifications for Maia and other accelerators are compared in our AI chips table.