Microsoft’s new AI chip is all about saving money

Microsoft has introduced Maia 200, a new AI inference accelerator designed to run large AI models faster and more cost-efficiently across Azure.

Unlike training chips, Maia 200 is focused on inference, where models are deployed and used at scale.

Built on TSMC’s 3-nanometre process, Maia 200 supports low-precision formats like FP4 and FP8, which help improve efficiency without sacrificing performance.

Microsoft says this delivers up to 30% better performance per dollar compared to its existing systems.

It comes at a time when large companies are increasingly designing their own chips to reduce their reliance on AI titan Nvidia.

Although the new chips can’t compete with Nvidia’s upcoming state-of-the-art units, they will be significantly more cost-effective.

The chip combines high compute power with a redesigned memory system, allowing more data to stay close to the processor.

This reduces bottlenecks and means large models can run on fewer devices, lowering infrastructure and energy costs.

Maia 200 also uses standard Ethernet networking rather than proprietary interconnects.

Microsoft says this improves reliability, simplifies scaling across data centres, and reduces the total cost of ownership at cloud scale.

The accelerator is already being deployed in Microsoft’s US Central region, with more regions coming next.

It will power internal workloads, including Microsoft’s Superintelligence team, Microsoft Foundry, and Microsoft 365 Copilot, with broader customer availability planned later.

Microsoft is also previewing a Maia software development kit (SDK) to help developers optimise models for the new hardware.

Nvidia, you got too greedy. - MV