Q&A with Gojko Adzic on the Book Running Serverless

In the book Running Serverless, Gojko Adzic introduces the basic concepts of serverless, including detailed step-by-step instructions to get started on AWS, but he also goes beyond the basics and explains subjects like storage, session state, and event handling. The book is divided in three parts with the first part explaining basic development tasks. The second part is focused on working with AWS services, and the final part deals with the design of serverless applications.

InfoQ recently interviewed Adzic about his thoughts on the book.

InfoQ: Tell us a bit about the book.

Gojko Adzic: Running Serverless is a technical tutorial for developers and architects who want to learn about serverless applications, a new generation of cloud deployment architecture that I believe is truly game-changing. Moving MindMup from a container-based application hosting platform to Lambda, we reduced our operational hosts by two thirds, while significantly improving the speed at which we could launch new features. And we’re not alone. A study by the research company IDC published in 2018 reports that companies adopting serverless applications on average 'lowered operating costs by 60% and were 89% faster at compute deployment'. These benefits are so significant that I expect most organisations with a cloud footprint to have at least some part of their applications running serverless over the next few years.

To truly get the benefits of this new deployment approach, people will have to rethink how their applications handle sessions, storage, how they bundle code and break down tasks. I wrote the book to capture the knowledge we accumulated building MindMup on Lambda since 2016, in order to help others go through that journey faster.

Unlike other books that already describe Lambda functions, my book has a strong focus on the relatively new but very promising tool built by AWS, called the Serverless Application Model (SAM). Tooling around Lambda functions was a big pain-point until recently, and lots of opensource projects appeared to plug gaps in Amazon’s offering. But over the last year AWS caught up, making deployments much easier with SAM. Running Serverless is, as far as I know, the first book that documents nicely the development and deployment flow with SAM.

InfoQ: Why did you decide to write the book?

Adzic: The whole Lambda landscape is moving very quickly and isn’t exactly well-documented for beginners. There are lots of blog posts and technical snippets online, many of them already outdated, and I wanted to give people an up-to-date, consistent and authoritative guide on the current state of the platform. Also, lots of online articles focus primarily on the APIs and initial steps, but solid advice on how to compose applications and get the most out of the platform is scarce. I also wanted to give readers a good overview of emerging architectural patterns so they can get started quickly.

The book contains a step-by-step tutorial explaining both the technical increments and the thinking process behind each gradual improvement, so people can understand the benefits and apply the same approach to their applications.

InfoQ: Who are the intended readers, and what can they expect to learn?

Adzic: The book will be useful for two groups of developers and architects:

• People who have no previous experience working with serverless applications and are interested in learning about emerging cloud architectural patterns
• People who already work with Lambda using other deployment frameworks and want to learn about AWS SAM, the Serverless Application Model

I think the key learning points are these:

• How to design applications that get the most out of serverless architecture
• How to create auto-scaling web APIs
• How to handle background tasks and messaging workflows
• How to set up a deployment pipeline for effective team collaboration
• How to test and troubleshoot code designed for AWS Lambda
• How to inspect and monitor serverless applications

InfoQ: What do you see as the key advantages for organizations adopting Serverless?

Adzic: For me, it’s the combination of speeding up product iterations and reducing operational burden. For teams using Lambda, most of the traditional operational and infrastructural tasks are included in the price and handled by Amazon’s engineers. That frees up application developers to work on the parts that make their application unique and focus on providing more value. As I mentioned before, the IDC report claims that many organisations can almost double their delivery capacity this way.

Also, Lambda comes with a very interesting pricing model, charging for actual utilisation and not for reserved capacity. This means that people do not have to pay for expected operational volumes, and keep spare capacity reserved for traffic spikes. For many organisations, this means significant savings in operational and hosting costs.

InfoQ: What do you see as the key challenges?

Adzic: Applications built using Lambda are effectively highly distributed systems from the very start, even for a hello-world application. This brings all the challenges of designing for massive distribution to even very simple use cases. Primarily, managing security, state and process concurrency are going to be tricky for people not used to this way of working. Tooling is getting much better, and SAM helps significantly in that area, but there are still challenges related to tracking requests through the network of small services, debugging and testing that need to be addressed as the platform improves.

[Click on the image to enlarge it]

InfoQ: Do you think that Serverless will be mainstream in the future?

Adzic: Absolutely. The commercial benefits are so significant that most organisations with applications running in the cloud will start moving at least pieces of their systems into Lambda. Also, many platforms are already pushing Lambda as a way to integrate or customise various services, kind of a universal glue. So people will be forced to use Lambda functions because that will become just the way to integrate with a provider. For some nice examples, see Netlify Functions and Twillio Functions. Both those platform providers started allowing users to submit Lambda functions for customisation, instead of operating webhooks.

InfoQ: Do you see any potential reasons for hampering adoption of Serverless?

Adzic: There are currently some technical constraints, such as limiting tasks to 15 minutes and function deployment packages to 250 MB, which might prevent using Lambda for longer-running workflows. Also, the technical platform is really designed for throughput, not for latency. Tasks such as payment processing, interacting with web user requests and converting files are well-suited to Lambda now. Tasks such as real-time algorithmic trading are not.

InfoQ: Is the client’s (e.g. a web browser) role changing when using lambdas?

Adzic: There is an emerging pattern with Lambda functions to move from a traditional three-tier deployment architecture to a more thick-client approach, even with browsers. In usual three-tier applications, the middle layer deals the business logic, but also with security, workflow orchestration and sessions. Lambda functions are not really suited to session management and long-running orchestration. Business logic stays in Lambda functions, but the other concerns need to go somewhere else. With the AWS platform providing security, a relatively good choice for session management and workflows is to push them to the client completely. So client code is getting thicker and smarter.

It’s too early to talk about best practices, because the platform is still rapidly evolving, but I definitely think that this is an interesting trend that needs closer examination. Moving workflow orchestration and sessions to client code from the middle-tier and servers means that application operators do not need to pay for idle work, and wait on background tasks. With a traditional three-tier application, for example, letting a client device write directly to a database, or access background file storage directly is a recipe for disaster and a security nightmare. But because the AWS platform decoupled security from processing, we find ourselves more and more letting client devices talk to traditional back-end resources directly, and set up Lambda functions to just act on events triggered by back-end resources.

For example, the tutorial in the book shows how to build a file conversion system similar to how MindMup works. A client directly uploads a file for conversion to a storage system, which dispatches an event about a new file, and then Lambda functions act on that event and reformat or convert the file. The client code handles the workflow coordination and session management. Lambda functions act just as glue code to configure temporary security grants and to handle the actual business logic for file conversions. This results in much faster and cheaper processing than with a traditional three-tier model, where an application server would handle most of that process. Removing the middle tier from the equation makes the application faster and cheaper, which is the biggest benefit of serverless.

[Click on the image to enlarge it]

InfoQ: Your previous books like Bridging the Communication Gap and Specification by Example are about how organisations can improve on delivering software aligned with business. Running Serverless is a much more technical book; have you shifted focus?

Adzic: I’ve always had lots of different interests, and they kind of move in a spiral. Getting better at building things provokes more research around quality, which opens more ideas around product management, which I can then apply to build stuff in new ways, forcing me to learn more about programming and operations topics. As I learn about new things, I tend to take a ton of notes for reference. MindMup matured quite a lot over the last few years, and we experimented a lot with new services on AWS as they became available. I also had a blast building an initial version of Video Puppet on Lambda, so this book is effectively my set of notes from those two projects.

If a topic I’m interested in isn’t well-documented already, then I have an urge to write a book about it. Creating something long form, categorised and consistent about a topic that is not well-documented makes the books valuable for readers. I enjoy writing, and doing a book also forces me to clarify and categorise my knowledge and research a topic in-depth, filling in my own knowledge gaps.

SAM and Serverless are currently very topical and not documented well, so I wrote this book for people who are starting on the same journey as I did a few years ago, hopefully helping them skip my mistakes and learn about good stuff faster. I’ve been using Lambda for a bit more than three years now, so it was time to put that experience on paper.

About the Author

Gojko Adzic is a well-known authority on how to deliver software aligned with business needs. He has written several books, including Specification by Example (winning the Jolt Award for the best book in 2012), Bridging the Communication Gap and Impact Mapping. He is the winner of the 2016 European Software Testing Outstanding Achievement Award and is currently partner at Neuri Consulting LLP.