Agile software development can be done at scale with the use of technology like self-service APIs, infrastructure provisioning, real-time collaboration software, and distributed versioning systems. Lean can complement and scale an agile culture with techniques like obeyas, systematic problem-solving, one-piece-flow and takt time, and kaizen. Fabrice Bernhard spoke about how their company uses technology with lean thinking for doing agile software development at scale at FlowCon France.
The agile manifesto doesn’t apply to large organizations, Bernhard stated. Leaders looking for principles to keep their culture agile while scaling their software organization will need to look elsewhere. And unfortunately that "elsewhere" is now crowded with options called "agile at scale", many of which are very bureaucratic and therefore not in the spirit of the agile manifesto, he mentioned.
Agile can scale, Bernhard said; there are many examples of organizations that scaled while maintaining an agile culture. In the body of knowledge of lean thinking, they found the principles they were looking for to scale their organization while staying true to the agile manifesto.
In the book The Lean Tech Manifesto that Bernhard wrote with Benoît Charles-Lavauzelle, he explores principles, systems, and tools that lean thinking provides to extend the principles of the agile manifesto. He mentioned some examples:
- Value models, obeyas, and value streams, to scale "Customer Collaboration" by ensuring "Value for the Customer"’ becomes the North Star of the whole organization
- Systematic problem-solving with PDCA and 5S, supported by team leaders and enabled in our digital world by collaboration technology, to scale "individuals and interaction" and transform the organization into a "tech-enabled network of teams"
- Jidoka, dantotsu, poka-yoke, pull, one-piece-flow and takt time, to implement "right-first-time and just-in-time" and scale "working software"
- Standards, kaizen, skills matrix and communities of practice, to scale "responding to change" with "building a learning organization"
Bernhard mentioned that they felt that lean thinking didn’t fully explain how some large agile organizations were succeeding. He decided to explore how the Linux open-source project and its community scaled from 1 to 55,000 contributors, where they used technology to address the scaling issues that they faced along the way:
The first scaling crisis happened in 1996, when Linus wrote that he was "buried alive in emails". It was addressed by adopting a more modular architecture, with the introduction of loadable kernel modules, and the creation of the maintainers role, who support the contributors in ensuring that they implement the high standards of quality needed to merge their contributions.
The second scaling crisis lasted from 1998 to 2002, and was finally addressed by the adoption of BitKeeper, later replaced by Git. This distributed the job of merging contributions across the network of maintainers and contributors.
In both cases, technology was used to reduce the amount of dependencies between teams, help contributors keep a high level of autonomy, and make it easy to merge all those contributions back into the main repository, Bernhard said.
Technology can help reduce the need to communicate between teams whenever they have a dependency on another team to get their work done. Typical organizational dependencies, such as when a team relies on another team’s data, can be replaced by self-service APIs using the right technologies and architecture, Bernhard mentioned. This can be extended to more complicated dependencies, such as infrastructure provisioning, as AWS pioneered when they invented EC2, offering self-service APIs to spin up virtual servers, he added.
Another type of dependency is dealing with the challenge of merging contributions made to a similar document, whether it’s an illustration, a text, or source code, Bernhard mentioned. This has been transformed in the last 15 years by real-time collaboration software such as Google Docs and distributed versioning systems such as Git, he said.
Bernhard mentioned that he learned a lot from how the Linux community addressed its scaling issues. And where the first agile methodologies, such as Scrum or XP, focus on a single team of software engineers, lean thinking has been battle-tested at scale for decades in very large organizations, Bernhard said. Anyone trying to scale an agile organization should study lean thinking to benefit from decades of experience on how to lead large organizations while staying true to the spirit of the agile manifesto, he concluded.