AWS announced changes to how Lambda functions connect to resources in a Virtual Private Cloud (VPC). This change — using pre-created network interfaces instead of network interfaces created for each function execution environment — cuts out a major factor in "cold starts" for serverless functions.
Is this change what IOpipe's Austin Huminski calls "an inflection point for serverless adoption amongst enterprises who are VPC-heavy"? To be sure, AWS Lambda users have been able to connect to servers and services running in VPCs since 2016. Since all of the compute infrastructure for AWS Lambda runs in AWS-owned VPCs, connectivity to customer VPCs traditionally occurred through elastic network interfaces. These elastic network interfaces got created in the customer VPC for each execution environment. Lambda function code couldn't execute until these network interfaces were created and attached. As a function scaled, more execution environments required even more network interfaces, as represented in this image from AWS.
With this just-announced update to AWS Lambda, the connectivity architecture is simplified, as explained in an AWS blog post.
Starting today, we’re changing the way that your functions connect to your VPCs. AWS Hyperplane, the Network Function Virtualization platform used for Network Load Balancer and NAT Gateway, has supported inter-VPC connectivity for offerings like AWS PrivateLink, and we are now leveraging Hyperplane to provide NAT capabilities from the Lambda VPC to customer VPCs.
As represented in this image from AWS, the network interfaces to customer VPC subnets are now shared across AWS Lambda execution environments.
AWS says that the one-time setup of the shared network interface can take 90 seconds to complete. But because the network interface gets created when Lambda functions are first created or VPC settings updated, the connectivity-related latency during cold start approaches zero. The same goes for function scaling, which no longer requires new network interfaces for each new execution environment. Amazon's Chris Munns recently tweeted out a graph showing a noticable drop in latency after this VPC networking change was deployed.
As an aside, the "cold start" questions often comes up as a reason that serverless functions aren't suitable for a number of workloads. What factors into the delay between when a function is invoked and when the code actually runs? Amazon's Tim Bray points out that cold start latency comes from many places.
The time it takes to launch a function depends on how recently you’ve launched the function. Because if you’ve run it reasonably recently, we’ve probably got it loaded on a host and ready to go, it’s just a matter of routing the event to the right place. If not, we have to go find an empty host, find your function in storage, pull it out, and install it before we fire it up.
Bray then says that once a function is "fired up", there are language-specific initializations that might require compiling code or starting a VM. And then there's whatever your function does to load state and connect to dependent services. Regardless, it appears that AWS is making solid strides tuning Lambda to reduce the cold start time, as seen in this recent comparison of cold starts by programming language, over time.
Of course, customers can, and do, use AWS Lambda without also running VPCs. Lambda functions can, by default, access anything available on the public internet, including many AWS services. In recent years, though, AWS has steered customers towards VPCs for many classic workloads. By default, Amazon EC2 virtual machines run inside a VPC. The same goes for relational database offering Amazon RDS, data warehouse Redshift, and Paas-like environment Elastic Beanstalk. Huminski at IOpipe says the VPC model feels familiar to enterprises adopting cloud, while now being more amenable to modern runtimes like AWS Lambda.
For companies or organizations migrating from private data centers or still running hybrid infrastructure, VPC seems like a no-brainer. It looks, smells, and tastes like the isolated traditional server environment with barbed wire on top that IT is accustomed to.
...
While the new changes make it more conducive for developers to connect Lambda functions to VPCs, the basic architecture doesn’t change in terms of your VPC.
This capability is being gradually rolled out globally, with expected completion of the rollout in a few weeks. AWS updated their documentation to account for this change, and shared that there are no associated costs for this upgraded functionality. According to AWS, many aspects of VPC connectivity remain the same, including the need for IAM permission configuration and a required NAT gateway to connect to endpoints outside of the VPC.