Why Microsoft is Building its Own AI Chips

How Microsoft's Maia and Cobalt Chips Will Power the Future of AI and Cloud

Key Takeaways:

• Microsoft unveiled two custom-designed chips for AI and cloud workloads: the Maia 100 AI Accelerator and the Cobalt 100 CPU.
• The chips will power Microsoft's own AI services, such as Microsoft Copilot and Azure OpenAI Service, as well as enable customers to access the benefits of AI and cloud on the Azure platform.

Microsoft is one of the leading players in the field of artificial intelligence (AI) and cloud computing, offering a range of services and products that leverage the latest innovations and breakthroughs. 

However, to meet the growing demand for efficient, scalable and sustainable compute power, Microsoft realized that it needed to optimize and integrate every layer of the infrastructure stack, from silicon to service. That's why Microsoft announced two new custom-designed chips at the Microsoft Ignite conference: the Microsoft Azure Maia 100 AI Accelerator and the Microsoft Azure Cobalt 100 CPU.

Microsoft Azure Maia 100 AI Accelerator

The Maia 100 AI Accelerator is designed to run cloud-based training and inference for generative AI workloads, such as large language models, image synthesis and video generation. The chip is optimized for performance, power and cost, and can handle up to 100 trillion operations per second. 

Microsoft collaborated with OpenAI, one of its strategic partners, to ensure that the Maia 100 chip would be compatible with the most advanced AI applications and frameworks. The chip will power Microsoft's own AI services, such as Microsoft Copilot and Azure OpenAI Service, as well as enable customers to access the benefits of generative AI on the cloud.

Microsoft Azure Cobalt 100 CPU

The Cobalt 100 CPU is an Arm-based processor designed to handle general-purpose workloads with high efficiency on the Microsoft Cloud. The chip is based on the Arm Neoverse N2 architecture, which offers 40% more performance per watt than the previous generation. The chip also supports 128 cores, 8 terabytes of memory and 512 lanes of PCIe Gen 5 connectivity. The Cobalt 100 CPU will run a variety of cloud services and applications, such as web hosting, databases, analytics and machine learning. The chip will also diversify Microsoft's supply chain and give customers more infrastructure choice.

With the Maia 100 and Cobalt 100 chips, Microsoft is demonstrating its commitment to building the infrastructure to support AI innovation and cloud transformation. By designing its own chips, Microsoft can tailor everything from silicon choices, software and servers to racks and cooling systems to meet the needs of its customers and the industry. The chips will start to roll out early next year to Microsoft's datacenters, initially powering the company's services and later becoming available to customers through Azure.