Netflix Spinnaker: Enabling Global Deployments

Netflix has released Spinnaker, their continuous delivery platform, as an open source project. Spinnaker allows you to build pipelines that represent a delivery process and execute them to produce a deployment. It's a compatible replacement for Asgard, so an all-or-nothing migration to Spinnaker isn't required.

You start with the creation of a deployable unit (e.g. a JAR file or Docker image) and end with a deployment in the cloud. Cloud environments supported include AWS, Google Cloud Platform, and Cloud Foundry. Spinnaker typically gets invoked after the completion of a continuous integration job, but can also be triggered manually, from a cron job, a Git repository, or from other pipelines.

Spinnaker offers the ability to manage your server clusters. Its application view allows you to edit, resize, delete, disable and deploy new server groups, load balancers and security groups.

Spinnaker is composed of JVM-based services (powered by Spring Boot and Groovy), and an AngularJS UI.

To learn more about Spinnaker and its open source launch, InfoQ interviewed Netflix's Spinnaker team, including, manager of delivery engineering Andy Glover, and senior software engineers Cameron Fieber and Chris Berry.

InfoQ: Spinnaker was released over a month ago, what's the community's reaction been like?

Glover: The community uptake has been tremendous! Having a platform that out-of-the-box accommodates multiple cloud providers and an extendable model to participate within, means that we are one large community rather than a bunch of micro-communities focused on various forks. This has the benefit that everyone within the community can take advantage of various innovations. We've seen quite a few pull requests come already from new community members and I'm confident we'll see more and more as we continue to increase overall pluggability.

InfoQ: It seems many cloud providers recommend uploading a single artifact and then scaling up using their API or UI. How is Spinnaker different?

Fieber: Spinnaker recommends an immutable infrastructure style deployment and natively works with the image formats for the underlying cloud providers (AWS AMI, Google Compute Engine Images, etc). Spinnaker also does support an orchestrated code push via the Quick Patch stage that lets teams iterate quickly by pushing and installing software packages to existing instances to eliminate the wait time for a new VM. This usage is typically used for a quick deployment into a test environment to run tests, or to patch stateful services such as datastores.

InfoQ: Do you know of any users that deploy to multiple cloud environments with Spinnaker?

Glover: I am aware of at least one fairly well known company making use of multiple cloud providers, however, they'd prefer to remain unmentioned. I suspect there are others and as the community grows it'll become more and more obvious which companies are pursuing a multi-cloud strategy.

InfoQ: How would you compare Spinnaker with Heroku's Pipelines feature?

Glover: I'd say the biggest differences between Spinnaker and their pipelines are (i) Spinnaker supports multiple pluggable deployment endpoints like AWS, GCE, Pivotal CloudFoundry, etc and (ii) Spinnaker's pipeline model is extremely flexible with many different types of stages. Stages can also be developed and plugged into Spinnaker by the community. Heroku's pipelines are designed to deploy to Heroku and their pipeline model is quite rigid. On the other hand, Heroku's pipelines are command line driven. At this point, we haven't release a Spinnaker command line client.

InfoQ: Looking at the GitHub project, it looks like the "gate" project is written in Groovy and uses Spring Boot. Why Groovy over Java 8?

Fieber: Spinnaker was the follow-on to Asgard (and an internal tool called Mimir) that were both Grails apps. The team had great depth in Groovy but felt that something a bit lighter and more operationally focused than Grails was worth investigating. Spring Boot was a natural fit, and Groovy works well in that environment. By picking Groovy we were able to cherry pick battle tested AWS code out of Asgard and reuse it in Spinnaker.

InfoQ: Spinnaker's UI project, "deck" is written with AngularJS (version 1.4). Was this a pleasant experience?

Berry: Early on, it was. Angular was pretty stable when we started 18 months ago, and there were a lot of libraries (UI Bootstrap, UI Router, Restangular) that made it easy to build and iterate on the UI quickly. React was another exciting technology at that time but still pretty new and the conventions and patterns weren't as fleshed out as they were with Angular.

Eventually, it became painful. Some of that has to do with the scale of Netflix - we have applications that need to render thousands of elements on the screen at once, and Angular 1.x just isn't performant at handling that many DOM nodes. In those places, we rewrote the code in plain JS, and performed diffs in a pretty crude manner. It's adequately performant, even with several thousand instances being rendered at once, but the code is brittle, as is the case any time you go off the paved road with Angular.

The other difficult thing has been figuring out how to make the UI modular and pluggable, so that different cloud vendors could build the UI modules they needed, and external users could create custom pipeline components. We've done an okay job on both fronts - it's not terrible, but it's not great. We've stolen a lot of code and concepts from UI Router just to make it workable, but we haven't seen anyone outside the teams at Google, Microsoft, and Pivotal try to build any customizations yet. Someone certainly may have - we just don't have any visibility into it.

None of this is to say we regret picking Angular 1.x for the UI. At the time, it was the right decision for us, and, knowing what we know now, if we had to go back and start over 18 months ago, we would have written some things differently but probably would have still used Angular.

InfoQ: Do you plan to migrate the UI project to Angular 2?

Berry: We're leaning towards doing so, but probably won't start into a migration for another five or six months. Angular 2 is still in beta, and the tooling support isn't there yet. Most of the folks outside Netflix writing UI features are not front end developers, and we want to make sure it's easy for them to figure out how to build things and debug issues when they get stuck.

It'll be interesting to see how Angular 2 evolves over the next year and what kind of stories come out around migrating from 1.x to 2 - we're just playing it a bit conservatively and hoping to learn from others.

InfoQ: How has Spinnaker improved deployments at Netflix?

Glover: First and foremost, it's given everyone a standard platform for delivery. Spinnaker makes it easy to do so with high degree of confidence as often as a team needs. Netflix services across the board are deploying more frequently with greater operational resilience. Spinnaker integrates with a plethora of other services and tools at Netflix and thus makes these features easily available to everyone. For instance, we have an internal service called ACA (Automated Canary Analysis), which is owned by another team at Netflix; nevertheless, it's a native Spinnaker pipeline stage, which facilitates testing. Before Spinnaker, each team who wanted to leverage ACA had to figure out how to plug it into their pipelines; now with Spinnaker, there's a standard means to leverage ACA and consequently, ACA usage is growing by leaps and bounds, leading to greater reliability across our production stack in AWS. As new tools and services that facilitate better testing, telemetry, or operational resilience are created, they'll be plugged into Spinnaker where everyone can take advantage of them.

InfoQ: What's your favorite Spinnaker feature?

Glover: Spinnaker supports an expression language that enables you to parameterize pipelines. This allows you to create some pretty sophisticated and, most importantly, reusable pipelines that can promote, test, and deploy across the globe.

InfoQ: Anything else you'd like to add?

Glover: While Spinnaker was created by Netflix, it's a successful project because of the strong partnership and contributions from partners like Google, Microsoft, Pivotal, and Kenzan. Looking at where we are today and where we want to go in the future makes all of us really excited. Deeper container support, increased overall pluggability & flexibility, and UI/UX enhancements are all underway and we're excited about the growing Spinnaker community!

Greg Turnquist, senior software engineer at Pivotal, authored a blog post about how Spinnaker works with Cloud Foundry. We thought it'd be interesting to get the perspective of someone doing integration work with Spinnaker.

InfoQ: At what point would you recommend Cloud Foundry users look into Spinnaker for deployments?

Turnquist: Cloud Foundry support is on the master branch of Spinnaker with extensive features. We have recently embarked on beta level testing with an active customer. So I would say the opportunity for Cloud Foundry users, whether that is PCF, PWS, or any other certified instance of CF, is ripe.

If you find yourself spending too much time manually pushing new releases to CF, and instead want to focus on your pipeline of deployment, smoke testing, and verification, the time to dive into Spinnaker and squeeze out inefficiencies in your release process is now.

InfoQ: Was it easy to integrate Spinnaker into Cloud Foundry?

Turnquist: I'm not sure if "easy" is the right expression. It implies there was little work to integrate the two. A lot of time was spent by me learning the underlying concepts of Spinnaker and mapping them onto Cloud Foundry concepts. As I got my arms around things, I realized Cloud Foundry was a perfect fit with Spinnaker. I had to learn many aspects about CF's APIs (I actually work on the Spring team, not the CF team). But the more I learned, the better things fit together.

Being able to group versions of an app together for upgrades/rollbacks, load balancing between new and old versions, and being able to have development vs. staged vs. production instances is 100% supported by both Cloud Foundry and Spinnaker. It shows the strength and flexibility of Spinnaker's architecture as well as the power of Cloud Foundry as a platform.

InfoQ: Greg, what's your favorite Spinnaker feature?

Turnquist: I get the most "oohs" when I open the UI's pipeline editor and start making spontaneous changes while talking about it. On the webcast Cloud Foundry After Dark, I had a simple pipeline with one step: Deploy To Production. As I described it, our host Andrew began asking me to make tweaks to turn it into a Deploy To Staging -> Smoke Test -> Deploy To Production. I made the adjustments as fast as he could call them out. Then we ran it and watched the user friendly readout. The value of molding your processes that easily cannot be understated.

InfoQ would like to thank the Spinnaker team and Greg Turnquist for answering our questions. The source code for Spinnaker is available on GitHub. To communicate with the community, see its Slack channel, Spinnaker questions on Stack Overflow, or @spinnakerio on Twitter.