Microsoft Azure has recently announced the release of a brand-new Virtual Machine (VM) powered by NVIDIA’s top-of-the-line GPU, the H100. This new VM, ND H100 v5, is designed to accelerate generative AI workloads, such as the AI language model ChatGPT.
According to Scott Guthrie, Microsoft’s Vice President of AI and Cloud, the company has spent hundreds of millions building a supercomputer to support large-scale training workloads. Microsoft worked closely with OpenAI to learn what they needed in building their training environment. This collaboration led to the developing of special-purpose clusters that could support such workloads.
The ND H100 v5 VM is the latest innovation resulting from this collaboration. It offers on-demand sizes from eight to thousands of NVIDIA H100 GPUs interconnected by NVIDIA Quantum-2 InfiniBand networking. Customers can expect significant improvements in AI model performance over the previous generation ND A100 v4 VM.
The ND H100 v5 VM features eight NVIDIA H100 Tensor Core GPUs interconnected via next-generation NVSwitch and NVLink 4.0. It also offers 400Gb/s NVIDIA Quantum-2 CX7 InfiniBand per GPU and 3.2Tb/s network per VM for a non-blocking fat-tree network. This VM has NVSwitch and NVLink 4.0 that deliver 3.6TB/s bisection bandwidth across eight local GPUs in each VM. Additionally, it has a 4th Generation Intel Xeon Scalable Processor and a PCIE Gen5 host-GPU interconnect that delivers 64GB/s bandwidth per GPU. It also has 16 channels of 4800MHz DDR5 DIMM.
This new VM is expected to deliver cutting-edge AI innovations to enterprises worldwide. Ian Buck, Vice President of Hyperscale and High-Performance Computing at NVIDIA, said the ND H100 v5 VM would power a new era of generative AI applications and services.
The ND H100 v5 VM is available in preview starting today and will become a standard offering in the Azure portfolio. It will enable anyone to unlock the potential of AI at scale in the cloud. With this new VM, Microsoft is once again demonstrating its commitment to advancing the field of AI and making it accessible to everyone.
