The book Save our Scrum by Matt Heusser and Markus Gärtner provides advice for teams to implement Scrum. It explores what teams that are having difficulties doing Scrum can do to get out of trouble and how they can find better ways to use Scrum.
InfoQ readers can download a sample of the book and can use this coupon to get a 15% discount on Save our Scrum.
InfoQ interviewed Heusser and Gärtner about why they wrote this book and how they structured it, the knowledge level of people that are doing Scrum and "saving Scrum", pursuing business value, how Scrum fails, and adopting and tailoring Scrum.
InfoQ: What made you decide to write this book?
Gärtner: Back in 2014, we met with an editor from a major publisher, over lunch during the Agile conference. We came up with the idea for a self-help book on Scrum. We brainstormed about ideas for a title - and then it hit us that Save our Scrum would be both a funny and meaningful title.
Heusser: We were kicking around what might help the community the most, is it is a testing book? Both of us had a lot of energy for the entire delivery process. Markus had just earned his Certified Scrum Trainer; I had just taken my lean delivery course public. Both of us saw companies struggling with Scrum. Time after time. Partial adoptions, compromise - and also some good work. The "good work" of scrum actually makes the problems that always existed visible!(Chuckles). We thought we had life experience and ideas to help. I'm kind of into boats; we talk about "train wrecks" a lot. We figured we could do a nautical theme focusing on the life preserver -- not complaining about the mess, but helping to bail out teams in trouble. That's the story of the title; of the book. Basically - we saw a lot of companies failing with Scrum and wanted to help.
InfoQ: How is the book organized? Why did you structure it that way?
Heusser: We organize the book as four parts that readers can turn to based on their interest and need. We start with "what is Scrum" using our language and our style - but intended to be fully compatible with the Scrum Guide. You'll see Markus' hand there, along with the section that follows - the 'why' of Scrum. In part two, I focus on the four common problems the compromise Scrum Adoption - the Whirlwind, the Compression Problem, the Tailoring Problem, the Assumptions Problem - and some solutions. In part three we provide 'nuggets', small answers to specific questions and problems. We're finishing up part three now; we intend to write case studies that are meaningful (and don't put you to sleep) in part four. Beyond "we did scrum and it was awesome", we'll talk about some real quagmires and how teams worked their way out of them.
InfoQ: Tell me more about these 'nuggets'; what are they?
Heusser: Most of them are stories, like the mini-case study above, some are advice, some are a stand against common anti-patterns we see. We have two that focus on this mythology around tasks and the forty hour work week. Personally, I don't get it. If you can make stories have similar size and count the number you do every week, or at least count the relative-size work effort ("points" or "velocity"), then why create an entire other system of measurement where you break the stories down into tasks, then plan to get the tasks done in a way that adds up to forty hours a week? The hours never match up, there are always unexpected and undocumented meetings, you end up using vague things like "admin time" to make the total forty, or else feeling bad and trying to change behavior or estimating method to get the two to match. They won't. Don't do that. We've found that looking at what you actually got done over time is a better way to predict the future than hoping and dreaming.
So that's the core of one nugget. Here's another, which I'll just cut and paste: "We Can't Do That"
Sometimes you hit a wall. The team tells you the architecture doesn’t support what you are trying to do. It is tempting, when you hear that, to change the business strategy based on the legacy software.
Other times it will be software. Often a single programmer, usually a junior one, says an approach is “too hard”, so you give up the feature.
At times like this, dig for a second opinion. As a different programmer what it would take. Be very pointed with the language - don’t ask if they could do it, ask how long will it take. This can reach the point of absurdity, where a single feature “takes longer” than a system rewrite. That is okay. Once you get a number on the table you can deal with it.
Let’s talk about a example. Say you have a system that takes a userID and a date and returns with information if that user has insurance coverage on that date. The system is not fast; it takes about a second. The product owner wants to enable on-line lookups, where the user uploads a file of 100-10,000 members, looks them up, and gets a file back, all within 5 seconds.
The simplest way to implement this is to read through the file, create a for loop and write out the answers to another file - taking between a minute and over an hour. That approach isn’t going to work.
The simple answer is “impossible” or “processing could take an hour.”
But processing might take an hour. You could pop up a screen that says the code is processing and then email the results. Or, better yet, create a different lookup function that takes an array of userIds. These possibilities are discovered through conversation.
Simple yes or no answers restrict possibilities. Avoid them.
Let’s turn up the heat a bit. Use the same system, but you want to put in the claimCode and the ProviderID to see if the benefit is covered. That is essentially claims processing, something you purchased your claims processing system to do, which runs overnight. The programmers don’t even know all the rules, which are complex.
If the product owner knows all the rules, the technical staff could create a program to do the lookup, saying “right now, it looks like this is a covered benefit.” (If coverage changes before overnight claims processing runs, all bets are off, of course.)
Or the company could ask the ERP vendor for support.
The point is to be open to possibility. “We can’t" shuts things down. We are saying — don’t shut things down. Allow people to explore. Take a break from judgement. Let the team self-organize, bouncing ideas off of each other. Let ideas percolate.
A final thought: If someone says that “can’t” do that, don’t give up. Ask what would have to change to make it possible. Then ask what it would take to make that possible. The answer might be “ten million dollars and a new claims processing system, a systems conversion, and four years of time …”
But it might not be.
InfoQ: How have you experienced the knowledge level of people that are doing Scrum?
Gärtner: My experience with people doing Scrum is very broad. Take for example my most recent engagement with a client that's building desks for the surgery room in hospitals. They are building a new desk including hardware, software, and the physical construction. I was very pleased with the Scrum Master there. He grasped the ideas behind Scrum very quickly. I was really amazed about the positive impact it had when they brought the whole team together in one room. Of course, they are at the beginning of their journey right now, yet they already experienced a few benefits like osmotic communication going on. I also sparked some ideas around extreme manufacturing to the people there, but right now this step appears to be too far off for most of them.
On the other hand, I have been involved with companies that are stuck in their current organizational design and component teams. Planning releases for customers with various features they asked for has been very tedious there. During my time at a client in the automotive business - about 9 months - I came to realize that the teams there were sub-optimizing since they gave up hope. They gave up the hope that the large organizational ship can be turned around. They gave up hope that their live can get better, and picked to optimize the factors they have an immediate influence on - with all the drawbacks this has.
And of course, I have seen teams and management doing lip service to Scrum - in my interpretation I claim that's because they didn't get the heart of Agile. I have seen teams doing three weeks Sprints, then stopping for a week to prepare for the next, just because their manager didn't believe that it's possible to prepare the next three week Sprint within the current one.
In all cases, though, I run into at least one or two persons that understand the underlying values and principles. Depending on their influence, they can be set out to save their very own Scrum implementations.
Heusser: Too many organizations have a handful of people that had a three-day class; everyone else had an one-hour introduction, or read a magazine article or a couple of blogs. For the most part people re-interpret the Scrum words in light of their past biases. Command and Control thinking is entrenched; it is going to take a lot more than that to get self-organization to take root.
InfoQ: What do you mean by Save Our Scrum, exactly? Does Scrum need saving?
Heusser: Oh boy. This title got us in trouble with everyone. Let's start out by saying that Scrum just doesn't have a huge successful adoption rate. Schwaber claims something like 75% of the organizations "doing" Scrum fail to have the results they'd hope for. The Scrum fanboys say that you can't blame the method if people compromise it so badly it is unrecognizable. People who think Scrum isn't enough, for example, perhaps Lean or XP proponents, will think that we should be talking about something else. Plenty of people we respect say that we should be writing something more open-ended, about chasing value or delighting users.
Here's what we see, though: Companies have made significant investments in Scrum, and are struggling. The struggle seems to be universal. When I gave some of the materials here in a talk, in the United States or in Europe, I tend to get three kinds of responses - people that are in waterfall or chaos shops, people that are struggling, and people who did struggle for a couple of years and are now on the other side of it. The first and last groups are the smallest, by far. The third group tends to empathize with the issues we talk about, and admit that they had to be worked through.
Our hope is to help people move to the third group a little faster, to share what we have learned. Looking around, we didn't see it much in the Scrum literature. Tobias Mayer's People's Scrum is probably the closest; a lot of that work is speculative thought experiments. It's good work, and worth reading. We found that we had tried many of those ideas, and the ideas had consequences. We see our work as sort of a follow-up, a bit more of a howto guide than The People's Scrum, sharing the attitudes of seeking truth with an awareness that different contexts might dictate different reactions.
The other thing (pauses) and it is hard for me to say this (pauses)... I haven't talked about this anywhere, really. But a lot of the "Scrum Literature" is on how to compromise. It is on how to manage global teams, how to deal with multi-tasking, or assigning a single person to multiple teams, how to deal with dependencies like a database administrator who works through tickets, so the team does not have everything it needs to deliver software end-to-end every sprint. I'm not sure if Markus agrees, but I thought it was worth writing a book that said "don't do that" -Sure, you can get advice on the best way possible to compromise, to play baseball with a ball of twine and only one base, but I thought the world could benefit from advice to stop trying to do the wrong thing righter.
Gärtner: Oh, I totally agree on that one. Besides the baseball reference. I am swimming guy. It's like trying to swim while one of your lower legs is pointing outside the water at a 90 degree angle. Depending on my training, I might do that, but I am quite sure not everyone else is capable of that, or would force them to do so.
Heusser: That's a longwinded way to say that yes, I think the state of the ScrumLand is not where we want it to be, and we were hoping, with this book, to move the needle, if not for all ScrumLand, then at least to make the world a little better for a handful of readers.
Gärtner: To me the answer to the title boils down to those folks I mentioned earlier. Those that got the whole idea of why we are doing Scrum. Some of them are successfully influencing their workplaces. These folks don't need saving. For the other ones, I would like to have not yet-another-Scrum-book out there, but one that can help them, help them save their Scrum.
InfoQ: Can you tell me a story about a team that needed saving. How did you save it?
Heusser: Let's be careful with credit. Coaches don't save teams; teams have to save themselves. What we can do is act as a sort of catalyst; to provide some insight, tell some stories, make some suggestions. Most importantly, we ask questions that can often reframe the problem.
One team I worked with was doing Scrum by the book. Not overly poorly, but there was some friction; they ended up getting the code 'done' on the second Friday, the testers would spend all weekend testing, and then find bugs. So the team "failed" to make the "sprint commitment" (now called the goal).
No one was happy.
The team also had a problem where the story would be complete yet would not fulfill the original mission. The email wouldn't get sent. The developers would say "oh no, that only submits the email to the email queue, it does not flush out the queue" - when the story was called "Send Email on Submit Button."
The "saving" here was not one single thing. There was a series, started before I got on-site (my colleagues, Matt Barcomb and Lanette Creamer worked with this team as well). Long term, we added work in progress limits (so stories didn't get dumped on test the last day), story kickoff that focused on goals, not just acceptance criteria, and, eventually, we dropped the whole idea of the sprint. This particular team was doing maintenance work on something like seventy systems. A "big" micro-project would be five or six stories batched together. The mean stories per production deploy was probably one point five and the median probably just one. We also got away from the fibonacci sequence - when I started we pointed stories to 1,2,3,5, and 8 points. When I left the team stories were 1,2, and maybe a task unworthy of a point. So the team could count stories, or velocity, and predict what they would get done, instead of looking at a list and nodding it was possible over two months.
That meant estimating capacity as a responsibility moved to project management. The team simply had to execute predictably, and project management could predict how much the team could deliver. Along the way there was a lot of code craftsmanship work, test infrastructure to be built, learning and adapting.
To recap: Crazy deadlines disappeared; crazy overtime disappeared. The team was able to move any piece of code from new-feature-tested to ready to deploy in the same day. The batch size got smaller. People had less meetings and spent more time on the things they enjoyed - solving business problems.
That's one example of life getting better; transitioning from chaos to Scrum to "Scrum plus" over perhaps a year and a half. That's much faster than other teams I've worked with. They really had the sense of continuous improvement. (In case you are wondering, Retrospectives and Planning moved to just in time. When people wanted to talk about something for retrospective, they'd put a sticky on the wall. When we got to six we called a retro. Or you could always call an emergency retrospective. With a team of twenty people, we figured we gained a person a person-year of effort by eliminating the "holy day" of meetings every sprint.)
Gärtner: I remember a client a few years back. I just came from a personal failure, trying to be Scrum Master for seven teams - never try that. They had one team doing Scrum for some time, but they felt something was not working. I facilitated their retrospectives, helped them during Sprint Planning, and convinced them that opening up the Sprint Review was a good idea. Skip forward two Sprints, and the team experiences in their Sprint Review how their feared Product Manager explained decisions in front of stakeholders. The relationship between the team and the Product Manager was very much improved by just opening up the Sprint Review. Within a few months, I was able to hand over more and more responsibility back to the team, and they were able to sustain their progress from that point on. That particular team needed saving, and I noticed it, and reached out for outside help at the right time.
InfoQ: Shouldn't we really be focusing on pursuing business value, not 'doing' Scrum?
Gärtner: In the context-driven testing community - where both Matt and me have been involved for quite some time - one of the principles is "The value of any practice depends on its context". Of course, we should be focusing on business value in some contexts. On the other hand I have seen contexts where it's really hard to grasp the business value. One recent training participant told me that they are working on a product that will eventually monetize in two to five years. Up until then they have enough funds, and the team totally doesn't know what will be of value in that time. In such a context, it's hard to focus on business value.
In such a situation I use Scrum to find out what is of value, and then shift towards focusing on that thing.
Another example: the surgery desk team had a challenge to tackle. During their second Sprint, they found out that the physical construction of the desk was not possible given the current parameters. They had some trade-offs to do in order to construct the desk so that parts of it would not collide with each other. They could reduce the lifting height, they could disallow horizontal tiltings up from a certain degree level or reduce the X-ray freedom for the final desk. All of these factors had consequences as they came from different markets, mainly the US, Europe, and Asia. Some trade-offs meant to upset the American market, others to upset one of the others. The team decided to go with the downsides for the American market. During the Sprint Review the product manager for the American was very upset with the decision, of course. But the team managed to describe the trade-offs to the stakeholders involved, and after only two weeks these folks found out something that would have taken months in their older approach.
So, focusing on business value? Sure, but what is business value? Does it suffice to calculate for the American market? It's really hard to tell, especially when you deal with engineering problems, and the trade-offs that stem for them. And it's certainly the same in software.
InfoQ: Can you give some examples of how you have seen Scrum fail? What was the reason that it failed?"
Gärtner: Scrum is a minimal framework. It tries to stay focused. For example, it doesn't say anything at all about development practices as it tries to stay open for applicability within various fields, not only software development. For example, there are folks that use Scrum in education, in hardware development, and building cars. So, you actually need to add some things to Scrum in your context, and that is by design.
Problems come up when practices that are not aligned with the Scrum values and principles are added. That's when people may be doing Scrum right, but end up with Scrum zombies carrying their bodies to the Sprint Retrospective. They are not doing Scrum wrong, but adding practices that undermine the whole ideas around Scrum, thus not getting the results they were expecting.
Matt: I saw retrospective notes from one client that was entirely blame-centric. "Bob's code stunk" (I'm pretty sure Bob was not in that day), "Sally can't make her deadlines", and so on. No concrete action items except maybe "work harder." You could argue that was a retrospective, but it was 'book scrum', that people were asked what went right, wrong, what should change, etc, and yes, they did give answers - but it's crap. That organization had a lot of command and control thinking, and a lot of fear. If people don't know something, then either the idea was bad or else they felt diminished. Amazingly, every new idea was bad!Shocking! (grins).
InfoQ: How would you describe Scrum?
Gärtner: In short, Scrum consists of three major roles, three major artifacts, and five activities, one of them we don't call a meeting. There are some duties for the Scrum Master, some for the members of the Development Team, and some for the Product Owner. Self-organization kicks in around the Product Backlog, the Sprint Backlog, and the Product Increment. We have a Sprint Planning meeting, a Daily Scrum meeting, a Sprint Review, and a Sprint Retrospective. In order to prepare the Product Backlog for the next Sprint, we need to spend some time on Backlog Refinement. That's the mechanics.
When you go out there, and try to implements the mechanics alone, chances are pretty high that you will end up with something we may call Scrum zombies. People will drag their mindless bodies to the meetings. That's not the space we would like to inspire, and that's certainly not the space that we want our readers to create.
If you look beyond the mechanics, the Sprint, the meetings, roles, and artifacts, you will need to realize that there is more essence behind the general structure that everybody these days seems to know already. Scrum is built upon empirical process control, and lends the three disciplines transparency, inspection, and adaptation from this one. Scrum is also based upon the previous work from Takeuchi and Nonaka, with its origin in the 1986 Harvard Business Review article "The New New Product Development Game". Nonaka and Takeuchi examined successful products in Japan at the time - real products like a computer, a printer, a copier, a car, and a bread baking machine - and they found that all of the companies they looked at produced radical successful products in a shorter timespan than their competitors by relying on cross-functional, autonomous, self-organizing teams.
We think that's where most Scrum adoptions these days start to fail: by recognizing the core behind Scrum, and getting the essence wrong. Or put another way: doing the same things, but expecting different results.
InfoQ: What are your opinions about tailoring Scrum? Do or don't?
Heusser: When Markus and I were at the Agile Conference kicking around this idea (and the year before) I made a real effort to talk to the non-coaches; to the contributors attending the conference. They all said they were doing "ScrumBut", "ScrummerFall", "Scrum-Ish", "Using Scrum as a Model to take ideas from", or, perhaps "Scrum Jemima", my term for when a team pours meetings on top of an existing process and calls it Scrum. All of them. Nobody was doing "book Scrum."
So everyone tailors it.
I see a difference, though, between compromising Scrum principles because they are too hard, or "won't work here", and actually understanding the system forces at play, making a studied and considered decision. The first leads to "ScrummerFall"; the second could lead to something more like "ScrumPLUS." A lot of our examples in the book, especially in the case studies, will go into things that are not book Scrum - the current version of the Scrum Guide says that the roles, artifacts and meetings are "immutable", and that, although it is possible to implement Scrum partially, "the result is not Scrum." So dropping Sprints, for example, would make the team "not doing Scrum."
I don't take that too seriously, as the Scrum Guide changes over time. So could you be "doing Scrum" one week, using the term "sprint commitment", then find yourself "not doing it" because the term changed to "Sprint Goal"? I guess. It doesn't really matter to me. What matters to me is if you understand the tradeoffs and are doing better due to inspecting and adapting ("Scrum Plus"), or if you are compromising because of "Political realities of the real world".
Gärtner: One lesson I learned from Craig Larman around "we do X, but..." often leading to stuff like ScrumBut, was to turn the question around from "but" to "and". "We are doing Scrum, but we skip the retrospective" then becomes "we are doing Scrum, and we currently don't know how to run the retrospective effectively every Sprint. That's why we will send someone to a training course/get an experienced facilitator/...". Problems start when making Shu-level but-statements. More often than not I find fruitful implementations when there are Ri-level and-statements with a perspective on why they are not doing a particular thing, or not doing it anymore, and what their plan for the future is. That's where main differences between successful Scrum and Scrum needing saving lie to me.
InfoQ: Do you have any insights into, say, how teams adopt scrum? Last words of wisdom for our readers on how to be more successful?
Heusser: There's a lot of 'shock therapy' out there. Flip a switch and suddenly everyone is 'doing' Scrum. That's an awful big-bang, large-batch-size, irreversible thing to do, when you think about it. It's a non-agile way to go Agile! Yes, I use terms like "go agile" or "do agile" for lack of a better term. I'm not sure that zen-like phrases like "Agile is something you be" are are that helpful, but I digress.
Gärtner: If you try shock therapy on a large project, it's possible the wheels fall off and people devolve to the old behaviors. Maybe you keep standups and sprints, but lose the rest. That's sad. Another possibility is it takes you three or four sprints to actually deliver any software. A lot of companies these days are doing an on-site Scrum class with two weeks of coaching, so the team has to figure it out -- and moving from six-month (or twelve-month) projects to two week releases to production is hard. Two ways to address that: You can crank the dials to eleven and do one-day sprints. You'll still get nothing until sprint four, but at least that is Thursday, when the coach is likely to still be around. What I really recommend is just starting with some training, then, as projects allow, doing retrospectives with teams, asking the team what they are willing to try next. Add scrum in a piece at a time. You will need to slice stories thin under almost cases, but let the team drive the rest -- to the extent that you can.
InfoQ: Your book is a work in progress. What will you add to the book in the coming months?
Heusser: As I said, we're finishing up the nuggets now, with a new push to production every week. When we finish the nuggets we'll do something like an affinity mapping to order them into chapters, so if you are struggling with, say, testing, you can turn to the testing chapter. After that we want to print case studies - but those are going to be hard. Many of our friends that have good stories to tell also have Non-Disclosure Agreements; some employees face so much red tape getting permission to print a story that they don't want to try. I am confident that we can get four or five brief case studies in the book that are fun to read, not fluff, that talk about real struggles and practical ways out of them.
Gärtner: Yeah, we have been a bit slow on the nuggets, lately though. We hope the pressure from this interview will convince us otherwise. :)
About the Authors
After fifteen years contributing and managing on software projects, Matt Heusser took his tiny concern, Excelon Development, full time in 2011. Today Matt serves as managing consultant at Excelon, where he does consulting, writing, training, and manages contracts. Probably best known for his writing, Matt is the managing editor of StickyMinds.com and a contributing writer at CIO.com. The lead editor for "How to Reduce the Cost of Software Testing" (2011, Taylor and Francis), he has served both as a board member for the Association for Software Testing and as a part-time instructor in information systems for Calvin College. Matt can be contacted at matt@xndev.com.
Markus Gärtner works as Organizational Design Consultant, Certified Scrum Trainer (CST) and Agile Coach for it-agile GmbH, Hamburg, Germany. Markus, author of ATDD by Example - A Practical Guide to Acceptance Test-Driven Development, a student of the work of Jerry Weinberg, received the Most Influential Agile Testing Professional Person Award in 2013, and contributes to the Softwerkskammer, the German Software Craftsmanship movement. Markus regularly presents at Agile and testing conferences all over the globe, as well as dedicating himself to writing about agile software development, software craftsmanship, and software testing, foremost in an Agile context. He maintains a personal blog at http://www.shino.de/blog.