Microsoft introduces custom chips to improve data center security and energy efficiency

At the Ignite developer conference today, Microsoft has unveiled two new chips designed for data center infrastructure: the Azure Integrated HSM and the Azure Boost DPU.

Scheduled for release in the coming months, these custom-designed chips aim to address the security and efficiency concerns in existing data centers, further optimizing their servers for large-scale AI workloads. The announcement follows the launch of Microsoft’s Maia AI accelerators and Cobalt CPUs and marks another important step in the company’s comprehensive strategy to rethink and optimize every layer of its stack – from silicon to software – to deliver advanced AI to support.

The Satya Nadella-led company has also outlined new approaches aimed at managing power consumption and heat emissions from data centers, as many continue to raise alarms about the environmental impact of AI-powered data centers.

Recently, Goldman Sachs published research It is estimated that advanced AI workloads will cause a 160% increase in data center power demand by 2030, with these facilities consuming 3-4% of global power by the end of the decade.

The new chips

While Microsoft continues to use leading hardware from companies like Nvidia and AMD, Microsoft has continued to raise the bar with its custom chips.

Last year at Ignite, the company made headlines with the Azure Maia AI accelerator, optimized for artificial intelligence tasks and generative AI, as well as the Azure Cobalt CPU, an Arm-based processor tailored to running general computing workloads on the Microsoft Cloud .

As the next step in this journey, the company has now expanded its tailor-made silicon portfolio with a specific focus on safety and efficiency.

The new internal security chip, Azure Integrated HSM, comes with a dedicated hardware security module designed to meet these requirements FIPS 140-3 Level 3 security standards.

According to Omar Khan, the vice president for Azure Infrastructure Marketing, the module essentially tightens key management to ensure that encryption and signing keys remain secure within the confines of the chip, without compromising performance or increasing latency .

To achieve this, Azure Integrated HSM uses specialized cryptographic hardware accelerators that enable secure, high-performance cryptographic operations directly within the physically isolated environment of the chip. Unlike traditional HSM architectures that require network roundtrips or key extraction, the chip performs encryption, decryption, signing and authentication operations entirely within the dedicated hardware boundary.

While Integrated HSM paves the way for improved data protection, Azure Boost optimizes DPU (data processing unit) data centers for highly multiplexed data streams corresponding to millions of network connections, with an emphasis on energy efficiency.

The offering, Microsoft’s first in the category, complements CPUs and GPUs by absorbing multiple components of a traditional server into a single piece of silicon – from fast Ethernet and PCIe interfaces to networking and storage engines, data accelerators and security features.

It works with an advanced hardware-software co-design, where a tailor-made, lightweight dataflow operating system enables higher performance, lower power consumption and improved efficiency compared to traditional implementations.

Microsoft expects the chip to easily run cloud storage workloads with three times less power and four times the performance compared to existing CPU-based servers.

New approaches to cooling and energy optimization

In addition to the new chips, Microsoft also shared progress made in improving data center cooling and optimizing their energy consumption.

For cooling, the company announced an advanced version of its heat exchanger: a liquid cooling “sidekick” rack. It didn’t share the specific benefits promised by the technology, but noted that it can be retrofitted into Azure data centers to manage the heat emissions of large-scale AI systems using AI accelerators and power-hungry GPUs like those from Nvidia.

In the area of ​​power management, the company said it has collaborated with Meta on a new disaggregated power rack aimed at improving flexibility and scalability.

“Each disaggregated power rack will feature 400 volts DC, allowing up to 35% more AI accelerators in each server rack, enabling dynamic power adjustments to meet the varying demands of AI workloads,” Khan wrote in the blog.

Microsoft is open sourcing the cooling and power rack specifications for industry through the Open Compute Project. As for the new chips, the company said it plans to install Azure Integrated HSMs in every new data center server starting next year. However, the timeline for the DPU rollout remains unclear at this stage.

Microsoft ignition runs from November 19-22, 2024