A post by Charity Majors, CTO at Honeycomb, reopened a debate over continuous deployment (CD) as she asserted that when people talk about CI/CD (continuous integration and continuous deployment) they’re only talking about continuous integration (CI), and that’s not enough.
The discussion covered not just its importance, but how many organizations are actually using it. Majors wrote that without CD, developers don’t get the timely feedback they need to evaluate and improve their code:
Until that interval is short enough to be a functional feedback loop, all you will be doing is managing the symptoms of dysfunction. The longer the delay, the more of these symptoms will appear, and the more time your teams will spend running around bailing leaks.
When teams batch updates and run deploys manually, Majors says, it leads to “the software development death spiral.” Without timely feedback, the time required to release new code increases while the process grows more difficult. So, teams need more resources to manage the added complexity. The best way to break or avoid this cycle, Majors asserts, is to use CD to break up the batches and create a faster feedback cycle.
Ester Peña, VP of software engineering at Travelers, emphasized the tight feedback loops with their take on how quickly code should be delivered to production:
Your apps should go from PR to production in 30 minutes or less.
Yes, you can. Yes, even in highly-regulated industries. I work at a Dow 30 insurance company, and we can.
Majors’s post triggered a spirited conversation on Hacker News. One thread opened with a commenter questioning whether it’s really true that only a few teams use CD:
I've been at teams and on projects doing CI and CD for the last... 15 years.
Sure, our deployment process was 10-15 minutes in the best of cases (because of serializing deployments for different branches), but where does the author's impression that the collective "we" are not doing CD come from?
Several responses to this thread asked if large teams could successfully negotiate the rapid code merges and builds required for CD. Others wondered if CD was possible in heavily regulated industries. But one commenter’s response raised the question of why one should implement CD in the first place:
Don’t do CD as a tech/ego benchmark thing, however (or Facebook/Google envy). Do it if you can agree that this brings some benefits to the business — e.g. reduced risk, higher throughput, etc. Also delivering faster really forces you to examine your process for bottlenecks and strip away overhead. I really like this line from the blog:
“The teams who have achieved CI/CD have not done so because they are better engineers than the rest of us. I promise you. They are teams that pay more attention to process than the rest of us.”
A few years ago, Jez Humble, co-author of the book Continuous Delivery, discussed some of the cultural and engineering bottlenecks to moving to faster code delivery in an interview with Software Engineering Radio:
... it’s a big cultural change for developers. But it's also a really important feedback loop because it means developers have to understand what it means to have production ready code, not just dev complete code. And that's super important for developers to actually understand that, because then you have to really think about these concerns like scalability and availability and writing log messages that are useful and not just pages and pages of exceptions, and things like archiving data. Developers have to build the software with these concerns in mind.
Pete Hodgson, an independent software consultant and speaker, argues that the effort behind CD might not be worth it, depending on an organization's needs:
It turns out, however, that getting humans out of the way is expensive. Fully automating your test and deployment processes requires a significant investment and one that is ongoing. Beyond test and deploy, there is also a need for sophisticated production monitoring and alerting, since you can no longer rely on an engineer simply keeping an eye on production metrics after they roll out a change.
While Majors’s post sparked a few spirited conversations, whether teams really mean both continuous integration and continuous deployment when they say CI/CD isn’t a new question. Anton Weiss, CEO at OTomato, wrote a post titled “Let's stop fooling ourselves. What we call CI/CD is actually only CI” in 2020. He made the same assertion as Majors, that few groups truly implement the CD part of CI/CD:
When asked what stops them from safely and regularly deploying every change into production environments - everybody seems to have their own reasons. Organizational, cultural, historical, technical, contractual.. Some go as far into denial as saying : "Oh, we don't need continuous delivery. In fact most companies out there don't really need it."
The debate over CD covers a lot of ground. While some argue over what “true” CI/CD is, others assert that it’s not practical or possible in some industries or situations.
