Microsoft Maia 200

What it is

Microsoft introduced Maia 200, an inference accelerator, on 26 January 2026 ^[1]. An inference accelerator is a processor built to run AI models that have already been trained: it holds the model's weights and carries out the arithmetic needed to produce an answer for each request. Training and serving a model stress hardware in different ways, which is why companies design parts tuned specifically for the serving side.

Design and memory

Microsoft said Maia 200 is fabricated on TSMC's 3-nanometer process and contains more than 140 billion transistors ^[1][2]. A process node describes a manufacturing generation at a foundry; smaller nodes generally allow more transistors and lower energy per switching operation on the same area of silicon.

The chip carries 216 GB of HBM3e memory running at 7 TB/s, plus 272 MB of on-chip SRAM ^[1]. High-bandwidth memory (HBM) is DRAM stacked in layers next to the processor so that data can be moved in wide, fast channels. Serving a model is largely a memory problem: generating each token requires streaming weights and cached attention state through the chip over and over, so memory capacity sets how large a model fits on one device and memory bandwidth sets how fast tokens come out. SRAM sits on the die itself, is faster than off-chip memory, and is small enough to hold only what is needed for very short periods of work.

Compute and interconnect

Microsoft said each Maia 200 chip delivers more than 10 petaflops at FP4 and more than 5 petaflops at FP8 within a 750 W SoC thermal design power ^[1][2]. FP4 and FP8 are number formats that store each value in 4 or 8 bits instead of the 16 or 32 bits used by conventional floating point; narrower numbers cut memory traffic and raise throughput per unit of silicon, at some cost in precision. The quoted figures describe the chip plus its on-package memory as a system-on-chip operating inside that power envelope.

Each accelerator exposes 2.8 TB/s of bidirectional scale-up bandwidth and supports clusters of up to 6,144 accelerators over standard Ethernet ^[1]. Scale-up bandwidth is the fast link that lets accelerators working on the same job exchange data directly, and it typically determines how large a single model can be spread across chips. Using Ethernet rather than a purpose-built interconnect lets operators rely on widely deployed networking equipment.

Microsoft said Maia 200 delivers 30% better performance per dollar than the latest generation hardware in its fleet ^[1]. Performance per dollar matters more in inference than in training, because serving runs continuously and cost is dominated by how much work a machine completes per unit of power and capital over its life.

Software

Microsoft previewed a Maia SDK with a Triton compiler, PyTorch support, low-level NPL programming, and a simulator and cost calculator ^[1][2]. A compiler translates model code into instructions the chip can execute, so support for common frameworks and compiler toolchains decides how easily existing models can be moved onto new hardware. A simulator and cost calculator let developers estimate how a workload would behave and what it would cost before physical access to the chip.

Where it runs

Microsoft deployed Maia 200 first in its US Central datacenter region near Des Moines, Iowa, with a US West 3 region near Phoenix to follow ^[1]. The company said the chip will run OpenAI's GPT-5.2 models and support Microsoft Foundry and Microsoft 365 Copilot ^[1].

Position in the lineup

Microsoft claimed Maia 200 has three times the FP4 performance of Amazon's third-generation Trainium and FP8 performance above Google's seventh-generation TPU ^[1]. Such comparisons come from the vendor introducing the chip, not from independent benchmark runs, and they compare peak numbers on specific numeric formats rather than measured throughput on real workloads. What the announcement establishes is that a large cloud operator now plans to serve a flagship third-party model family on silicon it designs itself, alongside a software stack and datacenter regions built around it.