Cliff Berg and Raj Nagappan on Agile 2: the Next Iteration of Agile

Transcript

Introductions [00:21]

Shane Hastie: Good day folks, this is Shane Hastie for the InfoQ Engineering Culture Podcast. I'm sitting down today with Cliff Berg and Raj Nagappan. You are two of the authors of Agile 2: The Next Iteration of Agile. Welcome, thanks for taking the time to talk to us today.

Cliff Berg: Our pleasure.

Raj Nagappan: Thanks very much for having us.

Shane Hastie: So Agile 2, why? We've got an Agile, it works. Or does it?

Why Agile 2 [00:46]

Raj Nagappan: I think it was quite timely, actually, to come up with another iteration of Agile, given that it's a 20th anniversary of Agile at the moment. I think that just passed last month or the month before. What's happened is that the world has moved on, so technology has moved on, business expectations have moved on and customer expectations have moved on. And I think that Agile really needs to catch up, it needs to iterate. So yeah. Agile is all about rapid iterations and the process itself, the philosophy itself, needs to iterate. And that's exactly what we've done here.

Cliff Berg: Yeah, I would agree with that, and it seems like there are really powerful things in Agile. When the Agile Manifesto came out, I was 20 years into my career. And I was CTO of a company at the time, a small company, about 200 people. But we built some pretty significant stuff. B2B stuff that had to work. And we got our feet into Agile very early, so we were early adopters. It was Extreme Programming, actually, in 2000. And then Agile kind of took the mantle away from XP and kind of co-opted it in a way. But there were some really powerful, good ideas that resonate, basically make things simpler. Smaller teams, work incrementally, build a little at a time. Kind of simplify.

My sister loves books on simplify. That's kind of what it was, simplify. Simplify for IT. And Raj was right, a lot of things have moved on and we have DevOps today which wasn't really possible 20 years ago because it really was commodity virtualization that made cloud services possible, which really is kind of what made DevOps possible. But some of the core things really resonate. But a lot of dysfunction sprouted within the movement, which is very common with any movement. It's normal. It's not really trying to say, "Ah, we did this bad," or something. This is like normal.

Some of the factors driving the need for course corrections in agile after 20 years [02:38]

Cliff Berg: And we feel that we need some course corrections and some of us also feel that the original ideas were really powerful but they weren't quite right. They were good, they were great actually, but there were some issues. Especially like if you look at the Agile Manifesto, some of the principles are a little bit extreme, and one thing that I found in my career I believe is that extremes usually don't work well. Unless you have an extreme situation. I'm not talking about Extreme Programming, that's a whole topic, we could do 10 podcasts on Extreme Programming, but just in general. Extremes. Whenever I hear something that's extreme, a red flag goes up.

So anyway, we feel that let's revisit some of this stuff. And we also kind of tested what people think. We found that there are a lot of people in the Agile community and the IT community, especially among programmers, who feel like there's some good stuff about Agile but there's some stuff that we don't like at all. So what is that?

Agile team rooms, the open plan office, just google that and you'll see how many people hate that. So there's some good things about it, it's not all bad. And some people love it and some people don't love it, there are differences too, and that's another thing. The Agile movement kind of became like one size fits all in a lot of ways, like this is how we do it. So it's been 20 years and we feel it's a good time to just revisit some of this.

Raj Nagappan: Yeah, I definitely agree with that, and picking up on some of the themes that Cliff has mentioned, so when the Agile Manifesto came out, one of its key strengths was its simplicity. And that's what enabled it to take off like a rocket, and simple messages spread very fast. The problem with simple messages though is that they're also open to interpretation, and we've seen that in the intervening 20 years, we've seen Agile interpreted in many different ways and misinterpreted in many different ways too.

So for instance, the manifesto itself says we value the things on the left and we value the things on the right. In a lot of misinterpretations people will take that to say okay, we value the things on the left and not the things on the right. That's like a common misappropriation. I think you can look up the Dark Agile Manifesto which says exactly that.

In itself, the Agile Manifesto was a great blueprint but I think it needed a bit more rigor around it, it needed a bit more fleshing out of well, actually, this is what we meant. We meant this and not that. So that's how we ended up coming up with a bit more clarification.

Shane Hastie: What is the clarification? What are the key, I suppose we could say, themes or dare we even say values from Agile 2?

Introducing Agile 2 [05:11]

Raj Nagappan: Agile 2 has values and principles the same way that the original Agile Manifesto did. The first difference you notice is that there's a lot more of them. So we have six pairs of values and we specifically say we value both of these things. So altogether, within those six pairs of values, you've got a dozen values altogether. And then in the principles we have 43 principles, I believe, and they're grouped into a number of groups, how many groups are they? 10 groups. So 10 different areas.

It looks like a lot but there's a lot of different ground that needs to be covered, because Agile software development, Agile development in general, whether it's for software or hardware or anything else, is quite a complicated process, it's quite a complicated beast. There are a lot of different things that you have to do.

But to come back to your question, I think probably the two most important themes that I would say that has come out for me from this process has been balance. So balanced the things on the left with the things on the right, that's critical. And the other one is leadership. I think that leadership has been sorely missing from Agile up until this point and a lot of the dysfunctions that we see are from disconnects between conventional leadership and how Agile teams want to work. So that has created a real schism or a real friction and Agile 2 has a real focus on effective leadership.

Shane Hastie: Let's explore those two themes. I see one of the values there, individual empowerment and good leadership. So individual empowerment, well, is it individual or is it teams? But there again there's another value, individuals and teams. Oh, this is getting interwoven.

Raj Nagappan: Yes.

Shane Hastie: So let's explore the leadership one.

Teams are made up of individuals – make sure you support them [06:45]

Raj Nagappan: Okay, so we've mentioned three different concepts here. So we've got leadership, we've got teams, and we've got individuals. And what happened in the original Agile is they basically talked about teams to the exclusion of the two others. But what you have to remember is even though there's no I in team, teams are made up of individuals. And if you don't also respect the aspirations and the requirements and the emotional needs of the individuals then your team's not going to be effective.

For example, in a sports team, you can take a whole bunch of world-class athletes and you can put them together in a team and the common saying is that okay, well if the team isn't coordinated then those athletes aren't going to perform well together. But the reverse is also true, which is that if you don't motivate those individual athletes to contribute to the team in a meaningful way and to feel comfortable, psychologically safe, feel like their aspirations and so on are taken care of, then they're not going to have that bipartisan relationship, they're not going to contribute to the team.

Coming back to the team versus the individual, so what that means is that you have to not only look after what the goals of the team are, to produce code, to produce output at the end of every sprint, things like that, but you also have to look after what the actual individuals need.

These might be things like career development, ability to focus deeply on their work, ability to care about their work. So not only collective ownership but to actually take a kind of possession where they look after it and they nurture it. Those kinds of things are important to them.

Then coming back to what you're saying about the leadership, so leadership is the one who has to empower that. So if leadership does not provide for those individuals to be nurtured and to be looked after, then you'll find that your team becomes shaky and the wheels will fall off the wagon eventually. And we do see that in quite a few dysfunctional teams where the individuals are unhappy, leadership is not looking after the individuals, they're just looking at the team level. Then everybody's locking heads with each other.

Shane Hastie: How does that good leadership show up?

Good leadership nurtures teams and individuals effectively [08:35]

Cliff Berg: You know, you have to realize we talked about this for almost a year before we put a bow around it, 15 people. Remotely, by the way, because it was during COVID. But how does leadership show up? Well, that's the rub. People have written about leadership for thousands of years, it's not a new thing, there are thousands, maybe millions, I don't know, books about it. First you have to decide what are your leadership models and how do you put it in place? It's just like self-organizing team but how do you create that team, who creates that team? Well, someone has to have some leadership to create the team in the first place.

We believe it takes leadership to actually properly constitute a team and to incubate it and watch it, especially if it's less experienced people or people with divergent ways of working. So deciding on what good leadership looks like, it probably looks like many different things. There isn't just one kind of leadership you need for most endeavours, you need several kinds of leadership. And to some extent you let it emerge but also you cannot just let it emerge because that can be very dysfunctional too if you just let leadership emerge. That can be just as bad as having autocratic leadership. It depends.

Raj Nagappan: That's a good point which Cliff raises, which is that if you don't appoint a leader then one will emerge from the group. And the person who emerges from the group may not necessarily be the most egalitarian leader, let's put it that way. They may well be someone with their own motives or with their own autocratic way of working and it just so happens that they're the most forceful in the group and that they force their will on to everyone else.

Different leadership styles needed in different contexts [10:20]

Cliff Berg: Yeah, and it might not be forced. Force might be soft force, it could just be like they're very charismatic. There was a study I saw some time back where it turns out that in remote teams, different people tend to emerge as the leader if you allow for emergent leadership. In remote teams, it tends to be like the organizer. The person who's moving stuff along.

Cliff Berg: Because think about it, people are sitting on their computers and there's no one to come around and say, "Let's all get together." So it tends to be the person who schedules a Zoom meeting at 10 o'clock tomorrow and creates an agenda. You know, the organizer. That person tends to end up becoming the default leader over time.

Whereas, in in-person teams it tends to be the person who looks like a leader. This is based on a study that I read, it tends to be the person who fits some kind of leadership stereotype. It's a different person.

Cliff Berg: I find that interesting, not so much because of the kinds of people who tend to get chosen but the fact that it's different, which kind of proves to me that the selection process for an emergent leader is not a trustworthy process. It can vary depending on whether people are in-person or not. And there also are studies of organizations that have very loose structure and interestingly, at the turn of the last century in the early 1900s, hierarchy, which we know can be very toxic, so I'm not advocating traditional hierarchy, but hierarchy was seen as a solution to lack of structure which tended to favor nepotism and the most aggressive and charismatic person.

Hierarchy was seen as the way that today we see Agile as the solution for hierarchy, 100 years ago hierarchy was seen as the solution for emergent leadership. So which is it? Well, it turns out it's neither. You don't want either one. You want something much more nuanced and sophisticated, and that's why leadership is the most complex issue of all in this, and what does good leadership look like? What are the many forms of good leadership? How do you get them to be creative? And that was your original question chain.

Cliff Berg: And that's the hardest question, how do you put them in place? That's cultural change. The Agile community is always talking about cultural change. So Agile coaches generally know senior leadership needs to demonstrate themselves the right paths. But they need to also decide on the patterns and getting that cultural change is the hard problem after you decide what kinds of leadership.

And there's more to it. I'm going to stop but there's more to it, because I mentioned hierarchy is not what we want but we do need hierarchy to scale. But today the issues tend to be cross-functional. They tend to cut across the org boundaries. So you need leadership in multiple directions at different times. So things have become very messy today. That's why it's so difficult.

Socratic leadership and the Coaching Kata [13:21]

Raj Nagappan: I'll give you two concrete examples of kinds of leadership that I think we have found in the group that have been quite effective. The first one is Socratic leadership, which is the idea of asking questions as opposed to dictating commands. That's based on the Socrates model of telling me why is this, tell me why is that, the five whys for example.

And getting the individuals in the team to basically, to get to the root of a problem by asking them to go through the process of identifying the root cause. So that tends to be quite a good, empowering way because it means that the individuals whom you're coaching, they're the ones who are actually coming up with the answers rather than you telling them.

A second way, which is a variant on the same thing, is the Coaching Kata. So the Coaching Kata from the Toyota lean process basically does exactly the same thing. You have a mentor and a mentee relationship. And the mentee is the individual whom you're trying to get to do the work. So you go through the process of coaching them and enabling them to come up with the answers themselves and to become more empowered. So you look at it more of a point of you as a coach rather than as a manager as such.

Shane Hastie: So that was leadership, and you're right, we can go on about leadership for a long time. But we'll point people to some of those leadership resources. Balance was the other important theme. Tell me about balance.

Balance is key -  every decision is a contextual decision [14:40]

Cliff Berg: Actually, Raj has talked about that a little bit and he mentioned the values. Actually, the other day on LinkedIn, someone looked at the Agile 2 values and basically said, "I don't like it because you're equating this with that." Because our values say, "We value this and that." And some of the things that we say we value sound kind of un-Agile. We value planning and contextual decision making, which I forget what it exactly is.

Raj Nagappan: Adaptability and planning.

Cliff Berg: Adaptability and planning. So his thought was you're not saying one's better than the other. And that was intentional and it's because it depends. It depends on the situation. You do need planning. You also need adaptability and sometimes you need more of one than the other. Generally in an Agile context we assume you want more adaptability and we generally do, but we don't want to throw planning out. Like Raj mentioned, the Dark Agile Manifesto is a parody of Agile and it basically says we value this not that.

You do need those other things. One of the major points of Agile 2 is we feel that the Agile movement was a rejection of trying to manage projects by numbers and by checklist. Not that checklists are bad, checklists are really important, but you have to know what you're doing. Every decision is a contextual decision. It's not like do this then this, there's not a procedure. Managing projects is not a procedure, it's a sequence of contextual issues that expose themselves over time. So it always depends. Malcolm Gladwell, the only right answer is it always depends.

So balance is about recognizing that every decision, an organization's trying to create products and do hard things. Every decision's a contextual decision. You cannot prescribe we should always do it this way.

Raj Nagappan: Yeah, I agree. I think with the planning and adaptability example, you can't do anything significant in life without creating a plan for it in some way. So for instance, if you've got a startup, if you're trying to raise money, you can't raise money without a plan. If you're trying to build a building, a house or an office tower or something like that, you can't build that building without a plan. You can't even bake a cake without a plan. I mean, have you seen an Agile cake? It might turn out great and it might be a complete flop, and that's the risk. So there is an old saying, which is that there's value in planning and it's certainly true.

When we look at how Agile swung the pendulum, they said okay, well we've gone way too far from the right, which is really, really concrete plans which go down into the hundreds and hundreds of pages, which we didn't end up following anyway. So let's throw that out and let's swing the pendulum all the way to the left and let's do no planning whatsoever.

More accurately, let's do very, very lightweight planning. So just two weeks at a time and no further than that. What we're seeing is that any broad scale initiative, you need to be able to plan further ahead than two weeks at a time. Maybe not in such detail that we were talking about earlier where you've got a 300 page plan, but you do need to have a sense of direction.

We see this not only with Agile too but also within the broader Agile community. You see concepts coming back in like product vision and product goals. So these are quite big in the product management area at the moment, and even the latest version of The Scrum Guide has added a product vision in there which it didn't used to have. So you see that balance actually coming back in the industry broader overall, not just in Agile 2.

These are not new ides – they are the distilled result of experience [17:55]

Cliff Berg: Actually, there's a point there I'd like to make that Raj kind of just alluded to. In Agile 2, there are no new ideas. There are no new ideas in Agile 2. We haven't talked about how Agile 2 was created but it was 15 very experienced people with a wide range of background, not just all programmers. It was product design, human resources, DevOps and programming of course, but other things. And the ideas of Agile 2 are ideas that we would hear and experienced ourselves when this stuff works.

If you talk to people who really do this stuff well at scale where they get complex products out rapidly and things are going well, this is generally what's in their head. This is how they look at it. So it's not like these are new revolutionary things. What we've tried to do is take all that wisdom that many Agilists have, including DevOps people, and Jez Humble, father of DevOps, always identified as an Agilist.

Taking those ideas from people who really know this shit. We've tried to add that stuff to the Agile narrative because it wasn't there. There have been very opinionated books, Klaus Leopold, Reinventing Agile, a lot of books basically saying, "Hey, we're doing it all wrong." But those are seen as opinionated and so some people love them and some people hate them.

We feel that the stuff that's the goal, what really works needs to be part of the standard narrative. So we're trying to enhance the standard Agile narrative with the wisdom and more nuanced understanding of these things.

Shane Hastie: One of the questions I had, because I was asked to look at this very early on, was why carry on calling it Agile?

These ideas are an iteration of agile approaches – not a replacement [19:40]

Cliff Berg: We debated that a lot.

Raj Nagappan: Yeah, so we had quite a few names which we debated and went back and forth with. One of the things is that there's significant brand value with the name Agile, and the subtitle of Agile 2 is the next iteration of Agile. We really felt that this was an evolution or an iteration on the original Agile concepts.

As we were debating the other different names, we have people who were involved in some of those other different movements, for example Agnostic Agile and so on. We always came back to the most powerful, simple name, the one which had the most impact and which people could get behind the most was Agile 2.

Cliff Berg: So we wanted to make it clear, we're talking about Agile, not some special type of Agile like XYZ Agile or something. We wanted to make it clear we're talking about Agile.

Shane Hastie: So if we come back to some of those values where you touched on adaptability and planning, individual empowerment and good leadership, individuals and teams. The next one in your chain there is business understanding and technical understanding.

Business understanding and technical understanding [20:40]

Raj Nagappan: So actually, I've got a very good example of this. A company I used to work for, they worked in the legal software space. And the thing about the legal software space is that you need to know your customers very well and their processes, their work processes are quite complicated and quite convoluted, and they're very high risk.

The problem is if you do something wrong as a software developer and you work with the data wrong, then the evidence which your software is being used to detect will be thrown out of court. And that's bad, right? So for example, if you're trying to prosecute a criminal and that criminal might be like cyber crime or organized crime or a drug dealer or something like that, arms dealer. These are like significant crimes that this software is used for. So that's a bad consequence of the programmer making a mistake, it's not just like it's a blue button rather than a red button.

What I found was, because I had worked for this company for quite some time, I got quite a good understanding of what the particular customers workflows were and what their particular needs were. And as the company got bigger and as we had more and more developers coming into the development group, they didn't come from that background. They didn't have that understanding. Because how many developers, how many software engineers have worked for a legal software company? Well, it's pretty small, it's a pretty esoteric area. There's not much of a crossover.

What happened was I ended up spending a lot of time coaching my newer developers as to what the actual business requirements were. Because they would look at a piece of paper, they would look at a requirement, and they'd say, "Okay, well I think I can do it this way and not that way." And I'd say, "Actually, no, you have to do it that way not this way because this is what's going to happen when it goes into the real world, when the lawyers use it, and when they submit it to court it has to satisfy all of these criteria."

That's a really good example where the technical side, where the developers and the designers and the testers, they have to have a very, very good business understanding of how the business works and what the business demands are for that product which they're making.

Then the same is also true on the other side, because I would talk with the marketing people and the salespeople and they would say to me, "Our customers are asking for XYZ, they want to have this particular feature, can we make it work?" And some things I'd say, "Yeah, okay, that's easy to do," and other things I'd say, "Well, that's not really possible, it's very, very difficult."

And what we see is that the best companies, you look at all of the Silicon Valley giants, the ones who really are kicking goals and kicking it out of the park, their business function and their technical function are very, very well in tuned with each other. It's like a yin and a yang diagram. The technology people understand the business, what it is that they're trying to sell, and the salespeople, the sales side, they understand the technology, they understand what's feasible and what's not feasible. So we can't split off things into different divisions, into silos. Those two parts of the business have to really work together. That's what I've found.

Cliff Berg: Yeah, it's true. I mean, most companies today, most companies that aren't the corner drug store operate on a technology platform. Whether it's a web platform or something non-computer but... Like Tesla, they make cars but their factories and design systems, that's their technology platform. Which have a very huge digital component to them.

And the value proposition today often is not just what do we sell but how do we make it? Because that affects how fast we can turn out new versions and things like that. Today, for a product manager to make good trade-offs, they have to make trade-offs between we'll add a new feature or improve our time to market by changing how we do stuff. Improve our time to market or shorten our time to market while retaining the same quality, by changing how we do it. That has business value.

Can we get time to market down from two months to three weeks? That has a lot of business value. But they have to trade that off with hey there's some new feature we could have. So to be able to make trade-offs, they kind of have to have some level of understanding, not just of the product but how it's made too. So they can have those discussions and make those choices.

Like Raj said, you cannot be in a silo, you cannot just say, "All right, that's technical, I don't need to know that." And the technical people also need to understand what the heck is this product, how are people using it? Otherwise they can't fill in the blanks. They're basically acting like robots painting by numbers. People need to understand the whole flow. Up to some level. Not the experts and everything but to be able to hold good conversations about the entire flow, today really essential for everybody.

Shane Hastie: So the next one in the list of values is the outcomes and outputs.

Outcomes and outputs [25:14]

Raj Nagappan: Yes, so outcomes are very fashionable at the moment. In the product management area in particular, a lot of people are talking about outcomes over outputs. So what's traditionally happened with Agile is that people have focused on the outputs. So Scrum is a very good example of this, where you have a sprint, your sprint lasts two or three weeks, and then at the end of it you have an output, you have a product increment. So this is a tangible artifact, a piece of software typically, which is something that is spit out at the end of the sprint which is the result of your work. That's an output.

But what people have found is that you can do all of that and you can do that sprint after sprint after sprint, but your customers don't necessarily get value out of that. You might have a lot of features which they don't end up using, if they use it maybe it's marginally or maybe they don't use it at all or maybe it doesn't live up to their expectations.

That's why the conversation shifted from outputs to outcomes. So outcomes would be that the customer is actually using the feature, that they're getting value out of it and that it's helping them to solve some problem in their life. I think Kathy Sierra, I think she talks about your customer doesn't value using your software, your customer values getting on with their life and achieving some goal in their life. Your software, your product, whatever it is that you're making is really just a means to an end for them to do that.

That's how the conversation shifted over to say okay, we value outcomes. But the balance comes back to say okay, well outcomes are only really enabled by outputs. You have to do both. So you have to both make a meaningful contribution to your customer's life, you have to make them be able to solve their task or to solve their goal, and you also have to provide the concrete, tangible product which does that. So both of those things matter. So coming back to the theme of balance, you can't have the outcome without the output being realized.

Shane Hastie: And the last one, thoughtfulness and prescription.

Thoughtfulness and prescription [27:06]

Raj Nagappan: I'll let Cliff go on that one.

Cliff Berg: Yeah, Raj is letting me go with that because he knows that's near and dear to me. In fact, there were two areas during the development of this where we had a lot of contentious debate. One had to do with soft forms of leadership and the other had to do with what effective collaboration looks like. So much of Agile 2, it depends. It depends on the individuals, it depends on the circumstances, but we felt that the Agile movement in reaction to horrible dysfunction in the 1990s where, Shane maybe you experienced some of this, where someone would hire a consulting company to create a requirements document, and then they'd give that to another company to create a design and then they'd give it to another lowest bidder to implement the design. Maybe you can do that for a bridge, but you can't freaking do that for software. You end up with garbage, people won't understand the design, because they didn't create it and there's no one to ask about the requirements because that consulting company left.

So basically it's like giving someone a big document on music theory and say, "Here, learn to play the piano." So we realized that the Agile community, in reaction to that, basically said, "Let's stop doing that, let's have conversations." Which is really powerful. That's what you need. But you also need documents too, but good documents, not big stupid documents. You need the right documents made by the right people who are still there and having conversations. The people doing the work should be creating the documents because writing something down is part of the learning process, it's part of how you figure out what it is you're going to do.

It's like we replaced one dysfunction with another dysfunction, and that comes into how do people collaborate effectively? One of the principles in the Agile Manifesto I actually think is wrong. All the others I think they're misinterpreted or taken to an extreme, but there's good stuff there. There's one that I actually think is wrong because it's expressed as an absolute. It says the best form of communication is face-to-face.

Well, it depends. That's true sometimes but sometimes that's not true. If you say it like that it's a false statement. Collaboration about simple things, yeah. Hey Joe, what's the data type of the customer ID field? Oh it's a string, okay, thank you. Yeah, face-to-face. Of course, you just disrupted Joe, he was deep in thought and now you just scattered his thinking. But you got what you wanted really quickly and really effectively.

But if you want to talk about something complicated, some people like to immediately sit down and start talking about it but not everybody. Some people say, "Let me collect my thoughts on that and I'll write down what I think and I'll send it to you and then maybe we'll get together tomorrow and talk." People work differently. And for complex things, you need to allow people time to mentally build their mental model. Effective collaboration involves listening, talking, reading, writing and thinking. Those five things.

And if you short circuit one of those you potentially derail people's process. And again, people are different. Some people like to write, some people like to talk. Their brain's different and that's okay. We need to try to meet people where they are to the extent we're able to. But the Agile community, not just in going too far with the face-to-face communication. Which is a good thing, face-to-face communication is a good thing, but we went too far with it. We made it an absolute, we made it an extreme. It's like always the default, always have a meeting, always talk.

We also, in order to encourage the always talk, we put people in rooms sitting side by side where they can't focus. I myself can distinctly recall being in such rooms where I'd be working on something that was complicated and someone will walk by and start having a conversation with someone else and I would literally stop thinking until they were done. And I would pretend I was still staring at the screen, I was pretending to be working but I literally completely stopped my thinking for 10, 15 minutes until they finished. Then I was so frazzled I would get up and go get a cup of coffee and come back and kind of rebuild the mental body I had. So basically they took half an hour away from me.

Raj Nagappan: That's the concept of flow, right? Because flow is very easy to break and can be very difficult to get back into. One of the arts of learning how to control your flow state is being able to get back into that quickly, right?

Cliff Berg: Well, to get back into it because what happens when you get into a state of flow is you build a very detailed decorated mental model of a problem and what you're trying to do, you're trying to solve the problem. So that means that there are conditions that you're trying to meet. And so you're testing your mental model against what happens if this happens? Inputs, basically, to see if the mental model predicts the outputs that you want. Does my solution actually work?

And if someone disrupts that mental model, basically parts of it fall off and then when they go away and you try to get back in the flow, you have to go back to that mental model and fix all the broken pieces. You have to refresh those parts of the model, think them through again. Then finally when it's all fixed, that's when you're ready to resume.

So depending how long the interruption is, and how distracted you get, the more distracted, the more interrupted, the more pieces fall off and the longer it takes you to rebuild the broken parts of that mental model.

The importance of concentration and flow [32:43]

Raj Nagappan: Yeah, I think that's one of the more important principles that we had in our list of principles, is you really need to give people the ability to focus and to not distract them. I think face-to-face conversations is one way that you do end up distracting them, but to bring that back to the face-to-face example, a simple example of that is how many times have you had a thought in the shower or a thought when you're going to bed or a thought when you're eating dinner?

So, they happen asynchronously, it's like what Cliff is saying about the mental model. When your mind is not consciously occupied on that problem, the cogs sort of start churning and they eventual come into place. So there needs to be a variety of communication mechanisms, not one solution fits all, and we really need to respect people to communicate sometimes but also to be able to maintain deep focus and concentration at other times. I think those two ideas go hand in hand.

Cliff Berg: This is actually an area where leadership, a particular kind of leadership, really helps. Because if you have a group of people, let's just say it's kind of random, or say you have a group of six people. Three out of the six have a general preference for let's always get together and talk it through. Maybe they're extroverts, I don't know what they are, but they tend to like to get together and always talk it through.

And the other three say, "Well I'm not ready to talk, I want to go and think about this. Don't bother me right now, I want to think it through. Maybe I'll write some things down but I'm not ready to talk yet."

Raj Nagappan: Right, so they'd say, "Let me come back to you on that."

Cliff Berg: Yeah, let me come back to you on that. And so if you have a group like that that are kind of like well what do we do? The people who want to talk it through will be confused and they'll think the others are being difficult or something like that. And the people who wanted to think it through are thinking that the other people are being aggressive and obnoxious and are not giving them a chance to like mentally process this.

So immediately you have a disconnect and there's no one orchestrating it to kind of show those differences. What can really help is if you have someone who is a Socratic leader who has a good skill of architecting good discussions. A discussion isn't necessarily a onetime event, it's over time, where they're sensitive to who tends to work in what way and who's asking for time, who's asking for a meeting, and then interpreting and suggesting.

I think that these three people need some time to mentally process. But they do want to talk about it and you other three, you want to talk right away. Bear with them, give them some time, and what if we all meet and talk about tomorrow at 11:00? Yes? Yes? Okay. So someone to kind of orchestrate that and find a solution that works for everybody and then be ready to pivot. Look at how it goes, see if they still need more time, see what comes out. Someone needs to be kind of watching that in order to make it go well. It might go well on its own, but it probably won't. But if you have someone who's good at that they can vastly increase the chance that it will go well and keep moving forward. And that's a form of leadership.

Shane Hastie: Thank you for that. One of the challenges, sometimes criticisms of the Agile Manifesto was it was the work of 17 middle aged white men getting together in a ski resort. Who was behind Agile 2?

Intentionally aiming for diverse viewpoints in the Agile 2 team [35:52]

Cliff Berg: We intentionally did not create a homogenous team. We actually created a table of the skills we wanted to cover, and it was important to have a global team, which we did. I mean, Raj is the other side of the world from me. I was joking to him that he's sitting upside down.

Raj Nagappan: And it's tomorrow.

Cliff Berg: Yeah.

Shane Hastie: That's right.

Cliff Berg: We have a very mixed team. And we have a range of skills. And the other thing is the authors of the Agile Manifesto, I think they had pent up thoughts. They had been emailing each other kind of ad hoc for a long time before they all got together. It was kind of a group of people, I think the ones driving it I think were the kind of people who want to get together and talk, because they didn't actually come up with anything until they met.

Whereas, if it were me, I would have tried to come up with it first and then got together in person to iron out differences. Because people work differently. But the fact that they didn't actually try to come up with something they agreed with until they actually met kind of tells me something about the people who were pushing that initiative.

But what they came up with was pretty freaking good, but most people don't realize they only came up with the values. They didn't come up with the principles in that meeting, they came up with the principles later over email, some of them emailing back and forth.

Raj Nagappan: That's actually true because that's documented in Bob Martin's book, Clean Agile, in the introduction. He does go into some detail about how, at Snowbird, they only created the four values and that the 12 principles were created by an email train back and forth in some weeks after.

That actually kind of negates one of the principles, which is that the best form of communication is face-to-face because the 12 principles of the original Agile Manifesto weren't created that way, right?

Cliff Berg: I think that that principle came from Alistair Cockburn, because he essentially had written that before the meeting in many of his blog posts. I think he might have had a book about it. But he had his famous chart and so on and then it ended up in one of the principles, essentially what he had been saying. That's the one I have the most problem with because it's an absolute.

But they came up with these four values, which were good, but yes it was a pretty homogenous bunch of guys. Most were from the US, there were a few from the UK and one from the Netherlands. And they came up with four values and then they went home. Whereas the Agile 2 team worked pretty diligently for more than six months and started from the premise of what's wrong and how do we fix it and then distilling consensus out of that.

The principles were what we got consensus on. So 15 experienced people all agreed to each of those 43 principles. That was a big lift, achieving that. We went through a lot of iterations, oh my god. We had Google Documents where we would have so deeply nested, because we couldn't meet in person, we couldn't even meet on Zoom because we're all over the world. A guy in Vietnam, Raj in Australia. So there was no way to meet face-to-face, it just wasn't possible.

We had to figure out how to do what GitLab likes to call asynchronous communication, and so we used every form available. We used Slack, we used email, we used Google Documents, and we had massively nested, complex, in like 20 different colors of back and forth discussions about different topics.

First we identified all the things that troubled us, and then we asked well why is that happening? What do we think? And then what should we do about it? Then finally we distilled the principles. Then finally we rolled those out and created a structure around it. Raj actually is the one who came up with the final structure. We had kind of a competition going.

So my point is that it was a major initiative. It wasn't a ski weekend, it was a major initiative doing that and we included in the content all of those problems and insights that we had along the way. Not everybody agrees to all of those. It made the list of a problem or insight if at least two people thought it was a problem or insight. But what we agreed on altogether were the principles, but we included everything. Basically you can see the why. For each principle you can see why do they think that? Then you look at all the problems and insights and then you can see why and then you can realize hey, that doesn't apply to my situation so forget that principle. Or now I understand what they're getting at, so now I can apply the principle intelligently. Because these principles aren't black and white. Maybe a little bit of this, a little bit of that.

Raj Nagappan: Yeah, so I think on the Agile 2 website you can click into each of those principles and it will take you to a much longer document and that longer document contains most, if not all of those points that Cliff was alluding to. So a lot of that stuff basically happened from many, many different back and forth conversations between two people or small groups of people. Then we took that and we nested it altogether and arranged it into groups and looked for common themes to drop out. And that's how the 43 principles came about.

So if you have some time it is well worth looking through, clicking into some of those principles, rather than just learning the one headline and looking into some of the actual problems and solutions and insights that led to the generation of that summary. Because some of them are quite deep, quite deep the insights which we've realized there.

Cliff Berg: It actually was a criticism. There were a few people when we published it who said, "Well, it's too long." So would you say oh the encyclopaedia’s too long? I mean, this is complex stuff. Building products is not simple stuff. There's on Agile coach who I know, I'm not going to name them, but very sharp guy. I just talked to him the other day but early on he said, "Well, the Agile Manifesto, it was short and emotional. I can get emotional about it, I can keep it in my head." I said, "Well, that's not what we tried to do. We're not trying to create something emotional. We're not trying to create a call to arms, let's throw everything out."

The Agile Manifesto was like let's throw out waterfall, good riddance. Burn waterfall at the stake, definitely. But we're not trying to do that, we're not trying to throw something out. We're trying to shore something up. So it's not emotional by design. It's not a short manifesto. It's a thoughtful treatise on this is how you make that work. These are the ideas that if you really drill in you discover these nuances and so here's food for thought. It's a lot of food for thought, is what it is. That's why it's not short by design.

Shane Hastie: Thank you very much for that, gents. It's been a really enjoyable conversation. If our listeners want to continue the conversation, where do they find this?

Cliff Berg: There's the Agile 2 website, which is agile2.net. We also wrote a book, which was a major undertaking, seven of us. There were 15 people who developed the Agile 2 content, but then we basically called out who has time and interest in creating a book version and so we created a book. The book is not a textbook, it's not like all the Agile 2 principles explained or something like that. It basically makes the case in a very readable way and kind of explains it. So it's a different kind of thing. So either one.

Raj Nagappan: There's also the LinkedIn group. We have an Agile 2 LinkedIn group where there's a community there and you can come and ask questions about it and gain further clarification. We've already had quite a few people come along there and post their problems as to how their current day-to-day workplace problems and said, "Well, how would Agile 2 handle this?" So that's a good resource to try. The Agile 2 community on LinkedIn.

Shane Hastie: And we'll include all of these links in the show notes. Again, thank you so very much for taking the time to talk to us today.

Cliff Berg: Our pleasure.

Raj Nagappan: Thank you very much for having us.

Mentioned

• Agile 2
• Agile Manifesto
• Dark Agile Manifesto
• Coaching Kata
• Kathy Sierra – creating passionate users
• Bob Martin – Clean Agile