Q&A on the Book Executive’s Guide to Disciplined Agile

The Executive’s Guide to Disciplined Agile explains how disciplined agile works at different levels in the organization. It provides a framework with principles and practices which helps you to streamline information technology and business processes in a context-sensitive manner.

InfoQ readers can download an excerpt of the Executive's Guide to Disciplined Agile.

InfoQ interviewed Scott W. Ambler and Mark Lines about the purpose and principles of the Disciplined Agile framework and the Disciplined Devops mindset, how you can pay down technical debt of legacy systems, how a Disciplined Agile Enterprise (DAE) looks like in practice, how to achieve success with an agile transformation, and what can be done to support continuous improvement in enterprises.

InfoQ: What made you decide to write this book?

Scott W. Ambler: We see a lot of organizations struggling with agile.  It’s fairly straightforward for a development team to become agile, but once you go beyond the bounds of software development it becomes very difficult for organizations to understand and then address the complications that they face.  We’ve been helping organizations for several years now to adopt Disciplined Agile strategies across all of IT and in some cases across the organization.  We felt it was time to share our experiences with a wider audience.

Mark Lines: This is our third book on Disciplined Agile.  Since we released our first book in 2012, Disciplined Agile has evolved.  Our intention is to update the content by writing a four book series to discuss Disciplined Agile from the various perspectives, namely an executive overview which is the subject of this first book, Disciplined Agile Delivery (DAD) for teams, Disciplined Agile for IT (DAIT), and Disciplined Agile Enterprise (DAE).

InfoQ: For whom is this book intended?

Ambler: In one word: Leaders.  This book is for people with the courage to look at the bigger picture, because frankly the big picture tends to be very ugly in practice. It’s for people willing to consider all aspects of their organization and who realize that IT is a key enabler of business agility. It’s for people who realize that context counts, that everyone faces a unique situation and will be agile in their own unique way, that one process does not fit all. It’s for people who realize that, although they are in a unique situation, others have faced similar situations before and have figured out a variety of strategies that you can adopt and tailor – you can reuse the process learnings of others and thereby invest your energies into adding critical business value to your organization.

Lines: Many try to simplify agile into an executive “summary” guide or whitepaper to dumb it down into a twenty minute read.  But the reality is that software development, DevOps, IT, and the way that organizations work are all complex topics.  So we have created a serious book for decision makers that explains how agile really works at different levels in the organization.  We believe that it is a very cohesive and coherent overview showing the interrelationship of the various disciplines across the enterprise.

InfoQ: What is business agility?

Ambler: Business agility requires an adaptive, lean, responsive, and learning organization. Business agility is something that emerges over time through lots of hard work – there are no shortcuts, silver bullets, or process frameworks that will solve your problems.  Business agility requires true agility across all of IT, not just software development, and a Disciplined Agile Enterprise (DAE) that is able to leverage that IT capability. And, because the environment in which your organization operates evolves over time, and because your competitors and partners also evolve, business agility proves to be a moving target in practice.

InfoQ: What is the Disciplined Agile framework and which purpose does it serve?

Ambler & Lines: The Disciplined Agile (DA) process decision framework provides light-weight guidance to help organizations streamline their information technology (IT) and business processes in a context-sensitive manner. It does this by showing how the various activities such as solution delivery, operations, enterprise architecture, portfolio management, finance, security, legal, and many others work together. The framework also describes what these activities should address, provides a range of options for doing so, and describes the tradeoffs associated with each option. In effect, DA provides the process foundation for business agility.

There are four levels to the DA framework:

1. Disciplined Agile Delivery (DAD). DAD addresses all aspects of solution delivery, from beginning to end, in a streamlined manner. This includes initial modelling and planning, forming the team, securing funding, continuous architecture, continuous testing, continuous development, and governance all the way through the lifecycle. The framework includes support for multiple delivery lifecycles, including but not limited to a basic/agile lifecycle based on Scrum, a lean lifecycle based on Kanban, two modern agile lifecycles for continuous delivery, and an exploratory lifecycle based on Lean Startup.
2. Disciplined DevOps. This is the streamlining of IT solution development and IT operations activities, and supporting enterprise-IT activities, to provide more effective outcomes to an organization.
3. Disciplined Agile IT (DAIT). DAIT addresses how to apply agile and lean strategies to all aspects of IT. This includes IT-level activities such as IT operations, support, data management, reuse engineering, and other capabilities.
4. Disciplined Agile Enterprise. A Disciplined Agile Enterprise is able to anticipate and respond swiftly to changes in the marketplace. It does this through an organizational culture and structure that facilitates change within the context of the situation that it faces. Such organizations require a learning mindset in the mainstream business and underlying lean and agile processes to drive innovation.

InfoQ: One of the principles of the Disciplined Agile framework is "Be Awesome". What does that mean?

Ambler & Lines: Who doesn’t want to be awesome? Who doesn’t want to be part of an awesome team doing awesome things while working for an awesome organization? We all want these things. Recently Josh Kerievsky has popularized the concept that modern agile teams make people awesome, and of course it isn’t much of a leap that we want awesome teams and awesome organizations too. Similarly the Poppendiecks observe that sustainable advantage is gained from engaged, thinking people. Helping people to be awesome is important because, as Richard Branson of the Virgin Group says, “Take care of your employees and they’ll take care of your business.”

There are several things that you as an individual can do to be awesome:

• Act in such a way that you earn the respect and trust of your colleagues – be reliable, be honest, be open, and treat them with respect.
• Willingly collaborate with others. Share information with them when asked, even if it is a work in progress. Offer help when it’s needed, and just as important reach out for help yourself.
• Be an active learner. Seek to master your craft, always being on the lookout for opportunities to experiment and learn. Go beyond your specialty and learn about the broader software process and business environment. By becoming a T-skilled “generalizing specialist” you will be able to better appreciate where others are coming from and thereby interact with them more effectively.
• Never let the team down. Yes, it will happen sometimes, and good teams understand and forgive that.

InfoQ: Another principle is "Enterprise Awareness". Why is that important?

Ambler: When people are enterprise aware they are motivated to consider the overall needs of their organization, to ensure that what they’re doing contributes positively to the goals of the organization and not just to the suboptimal goals of their team. This is an example of the lean principle of optimizing the whole, in this case “the whole” is the organization, over local optimization at the team level.

Enterprise awareness positively changes people’s behaviors in several important ways:

• They’re more likely to work closely with enterprise professionals to seek their guidance. These people – such as enterprise architects, product managers, finance professionals, auditors, and senior executives – are responsible for your organizations business and technical strategies and for evolving your organization’s overall vision.
• Enterprise aware people are more likely to leverage and evolve existing assets within your organization, collaborating with the people responsible for those assets (such as data, code, and proven patterns or techniques) to do so (one of the principles of the disciplined agile manifesto).
• They’re more likely to adopt and follow common guidance, tailoring it where need be, thereby increasing overall consistency and quality.
• They’re more likely to share their learnings across teams, thereby speeding up your organization’s overall improvement efforts. In fact one of the process blades of the DA framework, continuous improvement, is focused on helping people to share learnings.
• Enterprise aware people are more likely to be willing to work in a transparent manner although expect reciprocity from others.

InfoQ: What is the Disciplined DevOps mindset?

Ambler: There are several key aspects to the Disciplined DevOps mindset:

• One team. We’re a single team working together to support our stakeholders, not separate development, operations, data management, security, …. teams.
• You build it, you run it (and support it, and evolve it,...).  This focuses the team on building high-quality, maintainable, and supportable solutions.
•  Cross-functional people on cross-functional teams.  Building teams from cross-functional and generalizing specialists enables collaboration within a team because people have a greater appreciation for where others are coming from.  Cross-functional teams reduces hand-offs, thereby reducing the chance of injecting defects while reducing time to market and overall cost.
• Architecting for resilience.  Stuff happens in production - having systems that can ride out and even correct problems results in greater quality and operational success.
• Transparency.  Greater transparency within and between teams increases trust, enables collaboration, and enables effective governance.
• Continuous improvement. We should always strive to get better as individuals, as teams, and as organizations.
• Learn through experimentation. There is no such thing as “best practices”, only practices that work well in some contexts but not others.  The best way to discover what actually works for you is to try it, observe it in practice, and if possible measure it to determine if a practice or strategy works well for you given the situation that you face.
• Continuously streamline your workflow. No matter how good you are, you can always get better.
• Optimize the whole. Effective teams are enterprise aware, recognizing that they are part of a larger organization.  They should therefore strive to do what’s right for the organization, not just what’s convenient or “simplest” for them.
• Automate, automate, automate. ‘Nuff said!  Excelsior!
•  Eliminate technical debt.  We need to explicitly pay down, and better yet, eliminate technical debt if we intend to be responsive in the market place.
•  Reuse good stuff. An important strategy for decreasing time to market, improving quality, and to reducing technical debt (or at least not increasing it), is to reuse high-quality assets whenever we can.  Web services, microservices, frameworks, and many more are great strategies for doing so.

InfoQ: How can you pay down the technical debt of legacy systems?

Ambler: This is a big topic which we’ve written extensively about over the years.  In fact, I co-developed database refactoring which is the key technique for paying down database technical debt. 

In the Disciplined Agile Delivery (DAD) portion of the framework we explicitly promote the following strategies to address technical debt:

1. Do a bit of up front thinking to avoid injecting new technical debt.
2. Have an explicit architecture owner on your team who understands long-term implications. 
3. Be enterprise aware and do what’s right for your organization. 
4. Refactor technical debt away. 
5. Regression test continuously to identify new problems when they’re injected. 
6. Automate code/schema analysis. 
7. Measure technical debt so you understand the extent of the challenge. 
8. Explicitly govern technical debt so that senior leaders support your initiatives. 
9. Reducing technical debt should be part of your culture. 
10. Address technical debt before handing over an asset (particularly important when outsourcing)
11. Accept some technical debt, but do so explicitly.

A recent study we did looked at technical debt and agile teams.  We found that although 7 out of 8 teams were working with legacy assets in some way, that less than half of them were actively paying down technical debt.  This doesn’t bode well.

InfoQ: How does a Disciplined Agile Enterprise (DAE) look like in practice?

Ambler: Every DAE is different and constantly evolving.

Let’s start with several ideas that are foundational for a DAE:

1. Your organization and your people must be agile. Although our focus in this book is on effective process, the reality is that your organizational culture is the most important factor in your business agility success. You need people who strive to be awesome by collaborating in an enterprise aware manner to delight customers and continuously improve . Having said that, there is unfortunately little advice out there as to how an agile organization works as a whole, which is why in this book we choose to address that very topic. Coherence of your overall approach is critical – if one group of people is going in a different direction, if they aren’t working in an agile manner, they will drag down your entire organization.
2. It’s all about value streams. Your organization implements one or more value streams to provide value to your customers. A value stream is in effect a thread through your organization (and potentially partner organizations) of people collaborating to earn revenue. The best value streams delight customers, and part of doing that is to optimize the flow of your work to be reactive to customer needs.
3. There is no one right answer. Not only is every organization different, it is also evolving over time; Because context counts, you must identify an approach that works for you. Prescriptive methods are attractive because they appear to be something you can quickly learn and install. The reality is that you need to choose the strategies that reflect your actual situation to be effective – choice is good. And of course pragmatism should drive your choices, not a desire to “be agile.”
4. You need to sense and respond. Gone are the days of annual planning and three to five year roadmaps. DAEs follow an adaptive, outcome-driven approach based on experimentation and probing/sensing their environment and then responding.
5. You must be a learning organization. Everyone is responsible for learning and sharing their skills, regardless of the organizational domain that they’re working in. Because the modern marketplace evolves swiftly in unpredictable ways, you must be willing to experiment and to continually change your strategy to optimize flow. Furthermore, knowledge and knowledge work is becoming increasingly more important in modern enterprises, illuminating the need for continual learning.
6. Self-organizing teams need fast access to resources. Provide teams with the authority and responsibility to delight their customers. Senior leadership must help teams get the people, funding, help, and other resources that they require to react quickly to a marketplace opportunity. Provide boundaries within which to operate and an internal market for operational resources. Your objective is to match resource needs to prevailing customer demand – The implication is that some people may not be fully utilized, but this slack enables strategic work such as learning and improvement.

InfoQ: What's your advice for achieving success with an agile transformation?

Ambler & Lines: We’ve found that the following observations are critical:

1. Improvement is a journey, not a destination. Real improvement requires years of concerted effort, support, and guidance – not days of training, not weeks of coaching, and not (just) adoption of a new tool.
2. Many improvement journeys start as a transformation project, but morph into a long-term journey. In organizations that still have a project mindset it’s natural to mistakenly assume that you can simply run a short-term transformation project and “viola, you’re now agile!” With a project-based approach you are likely to have many false starts until you finally realize that improvement is a long-term proposition.
3. Every journey is unique. Every organization is unique, having its own challenges and its own priorities. You must tailor your approach to reflect the context of the situation that you face, one size does not fit all.
4. Your goal isn’t to be agile, it’s to serve your customers better. Many of your improvements will focus on becoming more agile or lean, but the real goal is to improve your value streams.
5. Improvement is hard and it requires change. If improvement were easy we’d all be perfect! Everyone, including managers and executives, must be prepared to adopt a continuous improvement mindset.
6. Prefer a pull over a push strategy. When change is forced on people, when you push it on them, they will resist and most likely subvert the change. Change is much more likely to stick when people identify challenges, identify potential improvements, and then willingly choose to experiment with (they pull into their process) the potential improvement(s).
7. Your improvement efforts need to address people (individuals and interactions), process, and tool issues simultaneously. Although you’ll invest a lot more effort in helping your people, in doing so you’ll find that you’re simultaneously also helping them to improve their process and to evolve the tooling required to support that process.

InfoQ: What can be done to support continuous improvement in enterprises?

Ambler & Lines: A lot actually.  In the book we work through how you evolve from a transformation strategy into a continuous improvement strategy. During the “transformation phase” you tend to have a significant investment in training, coaching, and education in order to kick start agile within your organization.  Then, once the new culture and ways of working take you, your focus shifts to a more self-sustaining, continuous improvement strategy that requires less (but still some) investment.

Strategies that support continuous improvement include:

• Experiments. Teams will need to run experiments to discover which techniques work for them in practice.  DA’s goal-based approach can help your teams to identify likely candidates to experiment with, thereby speeding up your improvement process, but you will still need to experiment to determine if a technique is in fact appropriate for you.
• Agile training.  You will find that you need to train new hires in agile, or give refresher training to existing staff.
• Skills training.  People will always require ongoing skills training in testing, collaboration techniques, programming, database development, finance, and so on. 
• Communities of Practice (CoPs)/Guilds. These are self-organizing groups where people help each other to learn and improve. 
• Agile/Lean Center of Excellence (CoE).  Although the need for agile/lean coaching will decrease after your transformation phase, you often find that new employees and sometimes even newly formed teams will need coaching from time to time.

About the Book Authors

Scott W. Ambler is the Senior Consulting Partner of Scott Ambler + Associates, working with organizations around the world to help them to improve their software processes. He provides training, coaching, and mentoring in disciplined agile and lean strategies at both the project and organizational level. Ambler is the (co)-creator of several agile methodologies and the (co-)author of several books.

Mark Lines is Managing Partner at Scott Ambler + Associates. He is an Agile Coach, co-creator of the Disciplined Agile framework, and co-author of several books. Lines is a “disciplined” agile coach, helping organizations all over the world transform from traditional to agile methods. He writes for many publications and is a frequent speaker at industry conferences.