This book will resonate at one point or another with anyone who's ever worked in IT. And it will resonate deeply with most. For some it will seem a plagiarism of their own professional story.
The growth of an organization naturally leads to separation of people and responsibilities into teams, sections, departments, units, which brings with it local specialization, work handovers across teams and formal change control procedures. Inevitably this will slow down the pace of IT changes in favour of the stability required for a medium to large enterprise. But is this really inevitable? And most importantly can business survive in the long term if it can’t make full use of IT’s capacity to enable business changes fast and reliably? This book answers these questions by showing that a truly collaborative approach between IT and business is possible.
We follow the story of Bill Palmer, unwillingly forced to take over the VP of IT Operations role at Parts Unlimited, a car parts company on the verge of financial meltdown. A company unable to cope with the advances their competitors are pulling out in the market. Even worse, they're having a hard time to comply with basic competences such as securing employees payments or keeping their inventory.
All too common problems the company is suffering from include lack of collaboration between teams, a blame culture, over dependency on individual heroes, imposing tools over collaboration, power games and pushing for individual goals/projects.
Granted, the novel draws an initial dark picture which is slightly overdramatized as are the ensuing 180 degrees turnaround (fueled by a mysterious khaki pants guru) to a rosy scenario filled with collaboration between all teams, continuous delivery and sky-rocketing sales increase. But this doesn't take away any of its value, instead highlights in a clear and compelling way the problems many organizations face today as their IT fails to support their business needs. And it shows another way is possible by promoting system thinking (over local optimization), feedback loops and a continuous learning culture.
Despite the DevOps reference in the title, breaking the divide between dev and ops is really just a small part of the overall cultural shift the book puts forward. The book addresses the divide between security compliance and IT, between dev and qa, between marketing and IT and finally the divide between business goals and IT delivery. DevOps on steroids one might call it.
By the end of the novel we’re reminded that DevOps doesn’t mean everything will work flawlessly with frequent deliveries, stable platforms and business-IT alignment. Instead it shows that with genuine collaboration in place everybody can give their best contribution to resolve problems and support the business in an agile and controlled fashion without resorting to the blame game.
InfoQ spoke with Gene Kim, one of the book co-authors, about the main motivation for this book, the state of IT in the industry, and other topics. We’re also sharing an excerpt from the book (first 170 pages) made available by the book authors. Gene has also posted about further related reading.
InfoQ: Your book expands the DevOps notion of Dev and Ops teams collaboration to include also business, marketing, QA and security teams, all working together to enable quick business decisions and implementation. In your experience how far are most organizations from realizing the need and/or benefits of this modus operandi? How important is the awareness of the DevOps movement in the enterprise IT world in order to go that extra mile and involve the business side in amplifying the feedback loops?
Gene: Regretfully, I think 5% or fewer of the organizations I've seen have reached the level of collaboration that are typified by the DevOps culture, where Dev and IT Operations, as well as Product Management, QA, Infosec and the rest of the business. The ones most widely cited are the famous examples of Netflix, Etsy, Facebook, Google and Amazon, where they've relentlessly examining how to improve flow through the product value stream to help the business win.
The hallmarks are easy to spot: they have Hack Days where they experiment, innovate, automate and help pay down technical debt; they are constantly innovating and automating work that impede flow or cause unplanned work; they attract talented people and empower them to take risks and go beyond the status quo.
All of these characteristics are equally applicable for any organization that needs to compete in the marketplace to win new customers and delight their existing ones. Arguably, organizations such as Facebook, Google and Amazon are "enterprises" in their own right. But, increasingly, we're seeing these types of success stories in financial services, higher education and retailing, such as BNY Mellon, World Bank, Paychex, Bank of America, The Gap, and Seton Hill University.
My genuine belief is that when these practices are widely adopted, we will see a surge in productivity that we haven't seen since the manufacturing revolution in the 1980s. The numbers we calculated show that there's nearly $3 trillion of value on the table that we could capture, if we could halve the amount of IT waste that could be mitigated with DevOps-style -- that's more than the entire economic output of Germany!
How important is this? When I think about the world my kids will inherit, and what this could do to standards of living, as well as quality of life in IT, I think this is incredibly, incredibly important.
InfoQ: The book suggests that successfully increasing collaboration and breaking silos is more likely to be driven by IT-focused bottom-up initiatives than a top-down business-focused understanding that IT needs to be agile enough to support quickly changing business needs. Would you agree?
Gene: I think widespread adoption of DevOps requires both bottom-up and top-down. My experiences with ITIL, IT operational process improvement and information security proved to me that bottom-up initiatives aren't often not enough. Process improvement must be explicitly and viscerally linked to helping achieve the highest organizational objectives. If it doesn't, the illusions of "faster time to market" and "ship more crappy stuff faster" will always win the day.
The core, chronic conflict that every IT leader faces is the need to simultaneously enable faster time to market (i.e., make as many changes as you can), while providing stable, secure and reliable IT services (i.e., make as few changes as you can). Until the seminal 2009 Velocity talk that John Allspaw and Paul Hammond did on doing "10+ Deploys Per Day: Dev and Ops Cooperation at Flickr," we thought that we had to choose one or the other. Allspaw and Hammond showed that we could actually get both, and get performance orders of magnitude better than we ever thought possible!
That's what every business and IT leader needs to know. That by doing this, they can get fast flow of features into the marketplace, while providing world-class stability and reliability, and this is how we help the business win.
InfoQ: What was the inspiration and the goal for writing this book? Is it based on the authors' experiences in IT?
Gene: Without a doubt, the inspiration for us has been Dr. Eliyahu Goldratt's seminal book, “The Goal: A Process Of Ongoing Improvement,” which he wrote in 1984. It’s a novel about Alex Rogo, a plant manager who must fix his cost and due date issues in ninety days, or his plant will be shut down. This book has been incorporated into many MBA curriculums and has influenced multiple generations of business leaders, and has sold over 6 million copies to date.
My co-authors and I studied this book for nearly a decade, getting ready to write “The Phoenix Project.” In many ways, I view our book as an homage to “The Goal.” We attempted to mirror most of the book structure and plot elements, using it as a vehicle to show the downward spiral that we've seen countless times over our careers.We constructed "The Phoenix Project" in a way that would help enable a shared understanding of the problem that affects all the silos, which inevitably leads to a downward spiral where Development and IT Operations can't meet their objectives and become locked in inter-tribal warfare.
In fact, the first 170 pages of the book is really designed to create the response of "holy cow, this is me that they're describing in the book," regardless of your role in the organization. Why? It's because it happens everywhere where DevOps practices and culture isn't embraced.
Why a novel? Studies have repeatedly shown that story-telling is the most effective mode of communications, and story-telling is what allows people to evangelize the concepts across and up the organization. One of my favorite reviews of the book came from Jeremiah Shirk from Kansas State University: "Some books you give to friends, for the joy of sharing a great novel. Some books you recommend to your colleagues and employees, to create common ground. Some books you share with your boss, to plant the seeds of a big idea. The Phoenix Project is all three.”
InfoQ: How would you sum up the book's message with a one-liner?
Gene: "The Phoenix Project" is a novel that first describes the problems that almost every IT organization is faced with, and then shows the practices of how to solve the problems, improves the lives of those who work in IT and be recognized for helping the business win.
InfoQ: You are working on the DevOps Cookbook with several other high profile practitioners in the field (John Allspaw, Patrick Debois, Damon Edwards, Jez Humble, Mike Orzen and John Willis). Is that meant to be a definitive hands-on book on how to go about solving the questions raised in "The Phoenix Project"? Do you have an estimated publishing date?
Gene: Let's say "Late 2013." :)
About the Book Authors
Gene Kim is a multiple award-winning entrepreneur, the founder and former CTO of Tripwire and a researcher. He is passionate about IT operations, security and compliance, and how IT organizations successfully transform from “good to great”.
Kevin Behr is the founder of the Information Technology Process Institute (ITPI) and the CTO of Assemblage Pointe. Kevin has twenty years of IT management experience and is a mentor and advisor to Chief Executive Officers and Chief Information Officers.
George Spafford is a prolific author and speaker, consulting and conducting training on strategy, IT management, information security and overall service improvement in the U.S., Canada, Australia, New Zealand and China. Co-author of “The Visible Ops Handbook” and “Visible Ops Security,” George is a certified ITIL Expert, TOCICO Jonah and a Certified Information Systems Auditor (CISA).