After one year as a virtual-only event, re:invent was back last week to Las Vegas with fewer attendees for the 10th edition, and with multiple sessions and keynotes, including a first one for the new CEO Adam Selipsky. AWS announced new features and improvements, with a focus more on packaged solutions than new primitives.
Acknowledging the over 200 existing services, the sometimes confusing API designs and the complexity of a large portfolio, Werner Vogels, CTO at Amazon, joked during his keynote:
You have asked for this, it is basically your fault!
Below is a review of the main announcements impacting compute, database, storage, networking, machine learning and development.
Compute
The main announcement was the Graviton3 chip, the next generation ARM processor with the preview of the EC2 C7g instances as reported separately on InfoQ. The preview of the M1 Mac instances was announced; they are built on Apple M1 Mac mini computers to create and test applications for iPhone and Mac, with similar prices and constraints as the existing x86-based EC2 Mac instances.
During his keynote, Selipsky announced the preview of the Trn1 instances powered by AWS Trainium. Optimised for high-performance deep learning training, they support up to 16 AWS Trainium accelerators and up to 800 Gbps of Elastic Fabric Adapter. Torsten Volk, analyst at Enterprise Management Associates, comments:
Trn1 and Inf1 do what the name suggests. Massive performance for machine learning training and inference.
New EC2 instance families have been introduced and are already generally available with different processors and configurations, including the G5g, M6a, Im4gn and Is4gen. For details, see a separate article on InfoQ.
As noticed by Scott Piper, security consultant, and Aidan W Steele, cloud consultant, AWS Lambda was supposed to provide direct invocation from a public URL, skipping the API Gateway and simplifying deployments. The feature was unfortunately rolled back and no announcement was made during the conference.
AWS Outposts, managed servers to run native AWS services on premises, are now available in different form factors, from 1U and 2U outposts servers to 42U racks. A final note on Kubernetes: Karpenter is a new open-source cluster autoscaler focused on AWS and supposed to replace the Kubernetes cluster autoscaler in most deployments.
Storage
As covered separately on InfoQ, Amazon FSx for OpenZFS is a managed file storage service for OpenZFS, the latest addition to the Amazon FSx family. A new EBS Snapshots Archive storage tier is now available to reduce the cost of storing archival snapshots.
S3 was revamped: the S3 Glacier Instant Retrieval storage class is a new storage class for rarely-accessed data with milliseconds retrieval. Mark Nunnikhoven, cloud strategist at Lacework, comments:
It snuck under the radar for most people but it could be quite a game changer.
The new bucket owner enforced option lets customers disable the ACLs associated with the bucket and the objects and S3 event notifications can now be used with Amazon EventBridge. The cloud provider has announced as well a free tier data transfer expansion, cost reductions for the Standard-Infrequent Access, One Zone-Infrequent Access and Glacier Flexible Retrieval storage classes and support for S3 in AWS Backup. See InfoQ's coverage of the S3 new features here.
Databases and Analytics
A significant part of Selipsky's keynote was on the importance of analytics services and how to improve their deployments using serverless options: Redshift Serverless, EMR Serverless and MSK Serverless are now in public preview and Kinesis on-demand is generally available. InfoQ covered separately the launch of Redshift Serverless and Kinesis Data Streams On-Demand.
There were high expectations about Aurora Serverless v2, the relational database that adjusts capacity in fine-grained increments and in a fraction of a second. Released in preview one year ago, the service is still not GA. Matthieu Napoli, PM at Serverless, tweets:
Now that re:Invent is finished, I guess it's time to get back to asking when will Aurora Serverless v2 be GA?
A few new v2 features, including support of RDS Proxy, MySQL 8.0 and PostgreSQL 13, were shown during the conference as reported by Goran Opacic, CEO of Esteh.
DynamoDB introduced a Standard-IA table class, a cost-optimised storage for infrequently accessed data. Amazon RDS Custom, announced in October only for Oracle (see InfoQ article), is now generally available for SQL Server. The new and more expensive option supports deployments with specific configurations and third-party applications that require customizations and access to the filesystem.
An interesting integration between existing services, Amazon DevOps Guru for RDS is a new capability to detect and resolve database performance issues on Amazon Aurora. Marcia Villalba, senior developer advocate at AWS, explains:
DevOps Guru for RDS uses ML to automatically identify and analyze a wide range of performance-related database issues, such as over-utilization of host resources, database bottlenecks, or misbehavior of SQL queries. It also recommends solutions to remediate the issues it finds. To use this capability, you don't need to be a database or ML expert.
Source: https://aws.amazon.com/blogs/aws/new-amazon-devops-guru-for-rds-to-detect-diagnose-and-resolve-amazon-aurora-related-issues-using-ml/
Networking and IoT
Selipsky announced the preview of AWS Private 5G, a managed service to deploy a private 5G network and connect thousands of devices and machines with low latency and high bandwidth.
In public preview, AWS Cloud WAN is a managed service to build and monitor global networks that connect resources across AWS and on-premises environments.
Already available in a subset of regions, VPC Network Access Analyzer is a tool to verify that VPCs meet expected network segmentation and network access requirements.
Available in public preview, AWS IoT RoboRunner is a robotic service that provides an infrastructure to manage robots from a single system view. The centralised repositories provide a unified data format for facility, robot, destination, and task data, helping enterprises building and deploying applications on fleets.
Source: https://www.allthingsdistributed.com
Machine Learning
Swami Sivasubramanian's keynote was focused on machine learning, with new services and features for SageMaker.
SageMaker Studio Lab, a free service to learn and experiment with machine learning based on open-source JupyterLab, is now in preview. SageMaker Inference Recommender is a capability of SageMaker Studio to automate load testing, optimise model performances, and reduce the time to get models from development to production. Converting models from their high-level language representation to hardware-optimised instructions, the cloud provider claims that SageMaker Training Compiler can accelerate the training of deep learning models by up to 50%.
SageMaker Serverless Inference is a new inference option to deploy machine learning models for inference without managing the infrastructure. Luca Bianchi, CTO at Neosperience, tweets:
This year's announcements raised the bar of serverless with some long-awaited releases such as serverless Kinesis, Redshift, and Inference. Three of my top picks.
Monitoring and Security
The two main CloudWatch announcements were Real-User Monitoring (RUM), a tool to understand performances and provide a better experience to end users, and Evidently, a capability to introduce experiments, feature flags, and A/B testing.
Amazon improved Inspector, a service that automates security assessments, adding automatic vulnerability management. Gert Leenders, cloud solution architect, comments:
The biggest gains from this announcement is that Amazon Inspector can now run continuously. On top of that, it automatically identifies resources instead of explicitly selecting resources for the Inspector to target.
New features were added to AWS Shield Advanced to automatically create, test, and deploy AWS WAF rules and mitigate layer 7 DDoS events. The CodeGuru Reviewer Secrets Detector detects passwords, access tokens and API keys in source code and configuration files.
Architecture and Coding
Vogels' keynote started with a timeline of AWS and showed some current metrics, before introducing new features and services for developers. Karan Desai, solution architect at AWS, tweets:
What is the most mind-blowing AWS service, you ask? It's not EC2. It's not S3. It's IAM. 500 million API calls every second... Imagine that.
Now in preview, AWS Migration Hub Refactor Spaces is a capability of AWS Migration Hub to refactor existing applications into distributed applications, usually based on microservices. Covered separately on InfoQ, the AWS Data Exchange for APIs enables customers to find, subscribe to, and use third-party API products from providers on AWS Data Exchange.
Designed to replace the AWS Forums and dubbed by many as AWS Stack OverFlow AWS re:Post is a managed Q&A service for crowd-sourced and expert-reviewed answers. Maurice Borgmeier, cloud consultant & trainer at tecRacer, reviewed the new service and comments:
re:Post seems a step up from the AWS forums, but as it is today there are lots of things missing to drive traffic away from the aws-* stackoverflow tags.
Source: https://aws.amazon.com/blogs/aws/aws-repost-a-reimagined-qa-experience-for-the-aws-community/
Released in 2017. AWS Amplify allows developers to build full-stack web and mobile apps. This year Vogels announced Amplify Studio to take the process a step further, adding a visual builder for all stages. Errol Hooper, AWS training architect at A Cloud Guru, wrote an article explaining why it helps startups and small business enthusiasts.
As previously covered by InfoQ, Custom Lenses is a new feature of the Well-Architected Framework. A new sustainability pillar was added to the AWS Well-Architected Framework to help companies learn, measure, and improve their workloads using environmental best practises. Jeremy Daly, GM of serverless cloud at Serverless, tweets:
New AWS Well-Architected Sustainability pillar. Translation: go serverless.
Amazon decided to target the mainframe world, announcing AWS Mainframe Modernization, a set of migration tools that include the ability to convert Cobol code to Java. Drew Firment, SVP at A Cloud Guru, highlights the significant market share:
1. Only 10-15% of total IT spend moved to the cloud
2. CSPs spending $100B+ on infrastructure this year
3. Mainframes still used by 71% of Fortune 500
The Cloud Development Kit v2.0 is now generally available, contrary to what Vogels showed during his keynote. The Amazon Builders' Library is a new repository of articles that describe how Amazon develops, architects, and operates technology.
Feedback and Recaps
During the event, AWS kept up-to-date the article announcements of AWS re:Invent 2021. Among other recaps, the ones from Resmo, Victor Grenu and Leenders are worth mentioning. Kristi Perreault's article focuses instead on the main serverless sessions. Vogels wrote some tech predictions for 2022 and highlighted a negative aspect of being back in person in Las Vegas:
After reading the horrific stories from Ali Spittel, Emily Freeman and Abby Fuller on their experiences from the past week makes me extremely sad. This was supposed to be a celebration of tech for everyone, but us "men in tech" still can't get our act together.
The dates of the 2022 edition have not been announced yet.
