Microsoft is entering the high-performance computing (HPC) market with their announcement of the general availability of Azure CycleCloud, a tool for creating, managing, operating, and optimizing HPC clusters of any scale in Azure. It is suitable for IT organizations of Azure customers, enabling them to create secure and flexible cloud HPC and Big Compute environments for their end users. Furthermore, Microsoft announced it would support NVIDIA GPU Cloud (NGC).
In August 2017 Microsoft acquired CycleCloud from the founders Jason Stowe, Rachel Christensen, Rob Futrick and Doug Clayton. Their company, Cycle Computing, was started in 2005 helped companies to do high-performance computing using Condor as an open-source scheduler. By doing big computing fast in the cloud, the company won several awards.
The company was also observing cloud vendors that could provide a robust infrastructure for their solution. James Stow said in a Bio.IT World article on the GA of CycleCloud:
Since we started CycleCloud over 10 years ago, the amount of compute power in a server has continued to exponentially increase. This is in part thanks to FPGAs and GPUs, and Azure has the broadest fleet of these accelerators.
IT administrators can quickly deploy high-performance clusters of computing, storage, filesystem, and application capability in Azure. According to the announcement, CycleCloud’s role-based policies and governance features make it easy for "customer organizations to deliver the hybrid compute power where needed, while avoiding runaway costs". Subsequently, end-users can rely on Azure CycleCloud to orchestrate their job and data workflows across these clusters. Customers can use start using CycleCloud by downloading the tool or through an ARM template.
Source: https://azure.microsoft.com/en-us/blog/microsoft-azure-the-cloud-for-high-performance-computing/
The support for the NVIDIA GPU Cloud (NGC) in Azure helps customers to accelerate their AI, and HPC workflows on a variety of virtual machines enabled with NVIDIA GPUs. NVIDIA provides a library of 35 GPU-accelerated containers for deep learning software, HPC applications, HPC visualization tools and a variety of partner applications from the NGC container registry. Customers can run these containers on the following Microsoft Azure instance types with NVIDIA GPUs, according to NVIDIA blog about the support of NVIDIA GPU Cloud on Azure:
• NCv3 (1, 2 or 4 NVIDIA Tesla V100 GPUs)
• NCv2 (1, 2 or 4 NVIDIA Tesla P100 GPUs)
• ND (1, 2 or 4 NVIDIA Tesla P40 GPUs)The same NGC containers work across Azure instance types, even with different types or quantities of GPUs.
The NVIDIA GPU Cloud Image for Deep Learning and HPC is available on Azure Marketplace.
Microsoft is not the only public cloud provider supporting HPC. AWS has GPU-powered EC2 instances available since October last year, which can be powered by up to eight NVIDIA Tesla V100 GPUs. These instances were designed to handle compute-intensive workloads ranging from machine learning to genomics. Besides Microsoft and Amazon supporting NVIDIA GPU’s, Google has their custom chips supporting customers to run machine learning workloads written for its TensorFlow framework. Each major public cloud provider appears to be pushing their cloud offerings towards the high-performance computing market.