Recently AWS announced the availability of new G5 instances, which feature up to eight NVIDIA A10G Tensor Core GPUs. These instances are powered by second-generation AMD EPYC processors.
Over two years ago, AWS made G4 instances available, which featured up to eight NVIDIA T4 Tensor Core GPUs designed for machine learning inference and graphics-intensive applications. In the new G5 instances, the company claims up to 40% better price-performance for inferencing and graphics-intensive operations than its predecessors. Furthermore, on the GPU side, the A10G GPUs deliver 3.3x better ML training performance, up to 3x better ML inferencing performance, and up to 3x better graphics performance than the T4 GPUs in the G4dn instances, according to the news blog post on the new G5 instances.
Source: https://aws.amazon.com/ec2/instance-types/
Each A10G GPU has 24 GB of memory, 80 RT (ray tracing) cores, 320 third-generation NVIDIA Tensor Cores, and can deliver up to 250 TOPS (Tera Operations Per Second) of compute power for AI workloads.
A respondent on a Hacker News thread on the new 5G commented on a similar offering of AWS with Nvidia V100 Tesla GPUs:
It's a cut-down and under-clocked A6000 with 24 GB GDDR6 instead of 48. So I would expect that it performs about the same as a V100 or a tad slower.
Customers can use the 5G instances to accelerate various graphics applications like interactive video rendering, video editing, computer-aided design, photorealistic simulations, 3D visualization, and gaming or for real-time AI inference performance at scale for use-cases like content and product recommendations, voice assistants, chatbots, and visual search. Furthermore, customers can also launch G5 instances in Amazon Elastic Container Service (Amazon ECS) and Amazon Elastic Kubernetes Service (EKS) clusters.
Other public cloud providers such as Azure and Google Cloud also offer a wide selection of instance types, varying combinations of storage, CPU, memory, and networking capacity, allowing you to scale your resources to the requirements of your target workload.
For accelerated computing workloads, Microsoft offers N-Series GPU-enabled virtual machines on Azure. These machines feature the NVIDIA Tesla accelerated platform and NVIDIA GRID 2.0 technology. And virtual machines available on Google Cloud for accelerated computing (A2) are based on the NVIDIA Ampere A100 Tensor Core GPU.
Currently, the new G5 instances are available in the US East (N. Virginia), US West (Oregon), and Europe (Ireland) regions in On-Demand, Spot, Savings Plan, and Reserved Instance form. The instances support Linux and Windows and are compatible with a long list of graphical and machine learning libraries, including CUDA, CuDNN, CuBLAS, NVENC, TensorRT, OpenCL, DirectX, Vulkan, and OpenGL.