During the recent re:Invent, AWS announced the preview of memory-optimized R8g instances powered by the fourth-generation Graviton processor. The new instances are designed for memory-intensive workloads, including databases, in-memory caches, and real-time big data analytics.
According to the cloud provider, Graviton4 processors provide fully encrypting all high-speed physical hardware interfaces and deliver up to 30% better compute performance, 50% more cores, and 75% more memory bandwidth than Graviton3 ones. The new generation R8g instances will be equipped with up to 96 Neoverse V2 cores, 2 MB of L2 cache per core, and 12 DDR5-5600 channels. Jeff Barr, vice president and chief evangelist at AWS, adds:
Graviton4 processors also support all of the security features from the previous generations and include some important new ones including encrypted high-speed hardware interfaces and Branch Target Identification (BTI).
According to the press release, the announcement states that Graviton4 instances can be 45% faster for large Java applications than the Graviton3 ones but does not provide further details or benchmarks., Liz Fong-Jones, field CTO at Honeycomb, had the opportunity to evaluate the new instances. She comments:
Our Go-based OpenTelemetry data ingestion workload required 25% fewer replicas on the Graviton4-based R8g instances compared to Graviton3-based C7g/M7g/R7g instances—and additionally achieved a 20% improvement in median latency and 10% improvement in 99th percentile latency.
The community's reaction has been mixed, with some users questioning the performance claims and others expressing concerns about the absence of pricing information. This is significant as Graviton3 instances have typically been more expensive than Graviton2 ones. User LunaSea comments on HackerNews:
Since Graviton3 still isn't available in most regions, especially on the RDS side, I'm really not holding my breath.
Graviton4 was not the sole custom-built hardware revealed by AWS during the conference: Trainium2, the second-generation accelerator for deep learning training of FMs and LLMs, is designed to deliver up to 4x faster training performance and 3x more memory capacity compared to the previous generation chips. Trainium2 will be available in Trn2 instances. Adam Selipsky, CEO at AWS, writes on X (formerly Twitter):
AWS’s chips were front and center at re:Invent where we announced Graviton4 and Trainium2. Graviton4 is the most powerful and energy-efficient chip we have ever built. Trainium2 will allow our customers to train their generative AI models even more quickly and with enhanced price performance. It was ten years ago that Amazon first decided to start developing its own chips. I love to present the challenges for software developers dealing with AI services using a simple example and service on AWS.
Multiple sessions on Graviton processors took place during the conference, with the recordings now available on YouTube. The c8g instances are currently available in preview.