I am privileged to be part of a team of experienced Lean and Agile coaches who get together regularly and share our stories, discuss what we have learnt and brainstorm new ideas. At a recent gathering last year we were exploring how to best deal with a request to provide training to a “Waterfall” team. It was a light-hearted discussion over lunch and my suggestion was a simple one; stand in front of the group and repeatedly say “No!” until the message is received and understood. That led to the creation of an “Introduction to Waterfall” training slide deck.
With a more serious hat on though, I don't really believe that is a good approach. While in the past I may have engaged with customers by saying “No!” to the existing approach, and immediately suggesting what I consider to be a better solution, I now try to understand the current situation first, before proposing a number of options and explaining the potential impact of each one.
Kanban has been the biggest influence on my current approach after hearing David Anderson talk about his approach in 2007. Since then he has gone on to document the Kanban Method as principles and practices in the book “Kanban: Successful Evolutionary Change for you Technology Business”. In the meantime I have developed an alternative model, which I call “Kanban Thinking” (See Figure 1). That name was chosen because the model has evolved as a way to represent the way I think about solving development problems using kanban-based techniques, rather than to represent any particular solution. As such, one of the goals in doing so has been to avoid being prescriptive, while still capturing the essence of much of what has been learned over recent years. The core components that enables this are the ideas that heuristics guide us in designing kanban systems, and that we can identify the impact of decisions to judge their validity.
Figure 1 - Kanban Thinking
Thus in response to the question in title of this post, the answer to whether kanban is common sense is “No!” However, common sense is an example of a heuristic, and I do believe kanban is a heuristical approach to solving the many challenges of product development. The remainder of this article will explore more about what heuristics are, the primary sources that led me to the realisation that heuristics are the right approach, and how we can use heuristics to design kanban systems.
Heuristics
The Merriam Webster online dictionary provides the following definition of a heuristic:
"involving or serving as an aid to learning, discovery, or problem-solving by experimental and especially trial-and-error methods ; also : of or relating to exploratory problem-solving techniques that utilise self-educating techniques (as the evaluation of feedback) to improve performance"
By this definition, we can see how heuristics are appropriate for situations in which there is no single or obvious right answer and in which we need to figure out for ourselves what to do. Thus if we think of kanban in terms of a set of heuristics, then it “can aid learning” of how to “improve performance” and we may get better results than simply copying good or best practice.
I have discovered a personal heuristic, which is that when I come across an idea or concept three times independently, I should pay attention to it. I'm undecided whether that is because three times means it must be something significant, or whether it just takes three instances before I wake up to the idea! I remember my discovery of XP being the first time I was aware if this. Heuristics is the latest example and these are the three instances of heuristics that made me sit up and take notice.
Heuristics Replace Rules - Dave Snowden
Dave Snowden refers to heuristics in relation to his Cynefin model (Figure 2), and in a blog post “Rules is rules” from earlier this year he said:
"we need a clear rule for when (or who) we can break the rules and heuristics that apply on the other side of the boundary. If you have to break the rules then that is OK, it will happen, but you have to then follow the heuristics."
Snowden makes a distinction between rules, which are appropriate for the ordered domains on the right, and heuristics, which should be applied for the unordered domains on the left. As an example, he cites the US Marines who have three heuristics for use when the original battle plan breaks down – capture the high ground, stay in touch and keep moving.
Figure 2 - Cynefin
In Cynefin’s ordered domains of simple and complicated, cause and effect are known or knowable. For simple problems we use a Sense-Categorise-Respond approach, applying rules based on best practice that is self-evident. For more complicated problems we use a Sense-Analyse-Respond approach, applying rules based on good practice that comes from expert knowledge.
In the unordered domain of complexity, however, systems are not causal but dispositional. That is, cause and effect are only retrospectively coherent, and cannot be repeated, but the overall impact will be similar. For complex problems we use a Probe-Sense-Respond approach, running small safe-to-fail experiments guided by heuristics to discover emergent practice. Further, for chaotic situations, we apply an Act-Sense-Respond approach, making quick decisions and inventing novel practice.
Given that we are often (although not necessarily always) designing kanban systems in response to complex problems, we can see that the use of heuristics is an appropriate one in determining what those emergent practices might be.
Snowden has proposed three general heuristics of his own for dealing with complex systems, which make them resilient to breaking and able to absorb the complexity:
- Distributed Cognition - having a network of connected parts, with knowledge shared across the network rather than centrally. Self-organising and cross-functional teams are examples of the application of this heuristic.
- Disintermediation - having direct access to data from its source, rather than through translation to interpretation. Information radiators and visual controls are examples of the application of this heuristic.
- Finely Grained Objects - having lots of small building blocks, which can be easily reconfigured, rather than having a single brittle entity. Small teams and small user stories are examples of the application of this heuristic.
Heuristics Support Substitution - Daniel Kahneman
In his recent best-seller "Thinking, Fast and Slow", Daniel Kahneman talks about the way we make judgements and decisions and defines a heuristic as
"a simple procedure that helps find adequate, though often imperfect answers to difficult questions. The word comes from the same root as eureka."
Thus heuristics are used to solve problems we have not solved before - you don't hear of anyone having a “eureka” moment about something they already knew. In particular, Kahneman says that we use heuristics to substitute difficult questions for easier ones, and suggests a number of specific instances.
One example is the Availability Heuristic. We use this to substitute a difficult question with one related to examples that are available for easy recall from memory. Kahneman describes one experiment where a group of people are asked, "Consider the letter K. Is K more likely to appear as the first letter in a word OR as the third letter". This is a question that most people don't know the statistical answer to, so we substitute the question "How many words can I think of with K as the first letter OR as the third letter". This leads most people to the conclusion that there are more words starting with the letter K, than having K as the 3rd letter, whereas statistically it is the other way around.
Another example is the Representative Heuristic, where we substitute a difficult question with one related to the resemblance of the subject to something we are more familiar with. Kahneman asked the following question "An individual has been described by a neighbour as follows 'Steve is very shy and withdrawn, invariably helpful but with little interest in people or in the world of reality. A meek and tidy soul, he has a need for order and structure, and a passion for detail.' Is Steve more likely to be a librarian or a farmer". Statistically there are far more farmers than librarians, however most people give the answer librarian because that is who they think the description most resembles.
The examples above have been chosen to demonstrate how heuristics can lead cognitive biases, where the substitution may result in an incorrect answer. The aspect of substitution remains a useful one, however, as we can see using the Marines’ heuristics mentioned earlier of capture the high ground, stay in touch and keep moving. These would naturally lead to questions about where the high ground is, how to stay in touch, with whom and what the next move should be. Similarly we can also this aspect to understand how we can use heuristics to guide the design of kanban systems. When responding to a difficult question regarding a complex scenario we can substitute alternative, easier questions, which while not guaranteeing a perfect answer, might help us come up with something good enough to improve.
Heuristics Guide Towards New Possibilities - Roger Martin
Design Thinker Roger Martin, in his book "The Design of Business", defines heuristics as
“rules of thumb that guide us toward a solution by way of organised exploration of the possibilities”.
He describes what he calls the 'Knowledge Funnel' (Figure 3) through which we move knowledge from having mysteries, to having heuristics, to having an algorithm. Thus heuristics frame the transition between exploration of the possibilities and exploitation of a solution.
Figure 3 - Knowledge Funnel
The transition should not always be in one direction down the funnel, however, as algorithms are based on the past, and the future is always changing and uncertain. Martin talks about the difference between validity and reliability. An algorithm that was reliable in the past, may not actually be valid, and thus may fail in the future. He tells the Bertrand Russell story about the chicken who predicts that he will always get fed when the farmer arrives in the morning. That prediction proves to be reliable, but is not valid, as the chicken discovers when the farmer arrives one morning with an axe. Thus heuristics provide a means to balance the need to for validity in exploring mysteries, and reliability in exploiting algorithms.
The search for validity, and the exploration of new possibilities, requires the use of abductive logic. Abductive logic, as distinct from inductive or deductive logic, is an idea advocated by Charles Sanders Peirce who said, “No new idea can be proved deductively or inductively using past data”. The differences can be demonstrated by using the Black Swan meme developed by Nassim Taleb, and expanded on by Jabe Bloom in a conversation with me earlier this year. Inductive logic is the logic of “what is”. If I have only ever seen white swans, I can say inductively that all swans are certain to be white. Deductive logic is the logic of "what must be". If I know that all swans are white, I can say deductively that a black bird will probably not be a swan. Abductive logic, however, is the logic of “what might be”. If I see a black bird that looks like a swan, I can propose abductively that swans could plausibly be black.
Referring back to the US Marine heuristics again, they are used to explore the mystery of how to win a battle, and come up with a new algorithm in the form of a new plan. Similarly, when designing kanban systems we use heuristics to explore the mystery of finding a valid solution to an organisational challenge, using abductive logic to discover and define the possibilities, leading to defining a reliable algorithm for that context. Heuristics may lead to surprising results from which we can learn, and as the context changes, we can return to them to help us continually evolve the algorithm to maintain its validity.
Kanban Heuristics
We have seen how the notion of heuristics is a powerful one when it comes to thinking about the challenges associated with product development. In fact, the Agile Manifesto can be thought of as a set of heuristics, with individual Agile processes and practices being specific algorithms. Kanban can also be understood as a heuristical approach, and the Kanban Thinking model shown at the start of his article includes five kanban heuristics that encapsulate the key areas I have found important to focus on, along with three impacts that encapsulate the areas of improvement I hope to make.
The heuristics are:
- Study the context
- Share the understanding
- Contain the work
- Sense the capability
- Explore the potential
The impacts are:
- Flow
- Value
- Potential
By applying these heuristics, and looking for these impacts, I hope to avoid the dogma of rules. Instead, I use the heuristics to respond to the difficult challenges we face by answering simpler questions as I explore the mysteries of product development and drive them down to unique algorithms.
Study the context
In “The New Economics”, W. Edwards Deming famously said, “there is no substitute for knowledge”. Studying the context is the means for acquiring knowledge about the current situation in order to improve it. This heuristic leads to substituting questions regarding what can be learned about the current situation. What is the customer purpose? What can we do to have empathy with our customer? What is the nature of the work? What can we do to analyse demand? What is the cost of delay for different demand? What is done to progress the work and where does the work get delayed? What artefacts and transformations exist as the work flows from the original idea to the eventual use? These questions lead to practices such empathy mapping, demand analysis or value stream mapping.
Share the understanding
Sharing the understanding of the current situation, typically in the form of a kanban board, creates an environment where people are more intrinsically motivated. In his book “Drive”, Daniel Pink describes intrinsic motivation as being created by autonomy, mastery and purpose, and a shared understand does this by enabling self-organisation (autonomy), improvement (mastery) and delivery (purpose). This heuristic leads to substituting questions regarding what can be done to ensure everyone has the same awareness and knowledge of the current situation. What is important for everyone to understand? How could that be modelled and visualised such that everyone can see, read and interact with it? I have a further set of heuristics that I use to help create a shared understanding which I call a visualisation TIP; Token, Inscription, Placement. How do we use tokens to convey information? What do we inscribe on those tokens to convey information? How can the placement of the tokens convey information? These questions lead to visualisations that are designed, owned and continuously evolved by the teams using them.
Contain the work
Kanban systems can be considered to be containers, with boundaries or loose constraints within which system evolves. Without such boundaries, a system will devolve into chaos. Containing the work, therefore, is the definition of work in process limits and other explicit policies that create those boundaries. This heuristic leads to substituting questions regarding the boundaries that can be defined to contain the work. How much work is in process? Where could work in process be limited? What work in process limits could be agreed? Where is the flow of work unstable or unpredictable? What explicit policies could reduce delays and improve the flow? Where could quality be improved? What explicit policies could help minimise mistakes or rework? These questions lead to decisions on limiting work in process, as well as policies such as ‘Definition or Ready’ and ‘Definition of Done’ which are entry and exit criteria for different stages of the process.
Sense the capability
As well as acquiring knowledge on what the work is, and how it gets done, it is also important to know how well it gets done. Sensing the capability of a kanban system involves getting a feel for its performance by both measuring and interacting with it. This heuristic leads to substituting questions regarding how to discover how well the system is currently performing. What impact are we currently having on the flow of work, delivery of value, or creation of potential? How can we measure productivity, responsiveness, predictability, quality, employee engagement and customer satisfaction? What insights would those measures give us? What regular interactions can help create a rhythm? When do we prioritise, replenish, plan, review and release work? These questions lead to establishing a cadence of meetings and events to progress work, and capturing metrics such as throughput, lead time, due date performance, production defects, employee retention and Net Promoter Score.
Explore the potential
Exploring the potential of a kanban system involves taking an approach of validated learning, applied to the creation of a new process rather than a new product or service. Thus it can be compared to “Lean Startup”, described by Eric Ries, who defined a startup as “a human institution designed to create a new product or service under conditions of extreme uncertainty”. Exploring the potential of a kanban system involves starting with the knowledge we have about the current state and experimenting from there. This can be contrasted with an approach of designing and defining a future state and trying to move towards it. This heuristic leads to substituting questions regarding how to continuously change and improve the kanban system. What changes could be made to have more of an impact on flow, value or potential? Which practices or policies could be amplified or done more of? Which could be dampened or done less of? What experiments could we run to learn more? How would we know whether we are improving? These questions lead to using approaches such as A3 Thinking, or Toyota Kata, which use the knowledge gained and shared from studying, to come up with policy changes as experiments, the results of which can be sensed to see if they have a desired impact.
Kanban Impacts
The heuristics just described, and the questions that they prompt, help us design kanban systems that can result in unique, contextual process algorithms. Teams regularly come up with different variations for their common situation, and I always tell them that all their designs are wrong – because there is no right answer and there is always potential for improvement. I have yet to work with a team that has designed a perfect kanban system at the first attempt. More important than knowing whether a kanban system is correct, is whether it is having the desired impact, and whether any exploratory changes are increasing or reducing that impact. These are the impacts that I look for in the Kanban Thinking model.
Flow
Making an impact by improving the flow of work is a result of ‘doing the thing right’, or improving efficiency. Limiting work in process, reducing batch size and eliminating delays will all enable flow through a smoother and more regular progress of work from initial concept to final consumption. Improved flow is likely to be a result of achieving the following outcomes:
- Greater predictability, because unnecessary delays and rework are avoided.
- Increased productivity, because more work is being completed.
- More responsiveness, because work is being completed sooner.
Using Roger Martin’s terminology from his Knowledge Funnel described above, greater flow leads to greater reliability.
Value
Making an impact by improving the value of the work is a result of ‘doing the right thing’, or improving effectiveness. Shorter lead times creating faster feedback will allow earlier validation. A managed portfolio of investments, limiting work through value streams, and using cost of delay to drive selection, will result in better economic outcomes. Improved value is likely to be a result of achieving the following outcomes:
- Greater customer satisfaction, because they are having their needs met.
- Increased productivity, because more valuable work is being delivered.
- Higher quality, because the work functions as intended and expected.
In Roger Martin’s Knowledge Funnel terminology, improved value leads to greater validity.
Potential
Making an impact by improving the potential of the people doing the work is a result of learning how to ‘do the right thing right’, or improving euphoria. Removing stress and overburdening from people’s lives and creating slack to build knowledge will result in greater possibilities for the future. Improved potential is likely to be a result of achieving the following outcomes:
- Greater employee engagement, because staff are motivated and happy.
- More responsiveness, because work can be committed to at a later point in time.
- Higher quality, because products and services can evolve more freely.
While Roger Martin doesn’t have an equivalent term for this, I propose that one fitting the rhyming pattern would be that improved potential leads to greater humanity.
Conclusions
I hope that this article has shown that kanban is not common sense, but like common sense, it is a heuristical approach which can help us think about how we might address the challenges of software and product development in organisations. I have described a Kanban Thinking model that encapsulates this approach with five heuristics - Study, Share, Contain, Sense and Explore – along with three desired impacts – Flow, Value and Potential.
To sum up, there are three recommendations that I hope you take away.
- Avoid the dogma of rigidly sticking to rules and following processes - instead use heuristics to tackle complex problems.
- Take the difficult questions those complex problems raise and use the five kanban heuristics to substitute simpler questions.
- When answering those simpler questions, embrace abductive logic to ask "what if?" - you may be surprised with what you learn.
This approach will lead to the development of a problem solving capability over the long term, rather than just a solution to the problems of the short term. Doing so will create a learning organisation that is sustainable, equipped to continuously balance validity and reliability, and able to enrich humanity.
As a closing metaphor, as someone who has to confess to watching Strictly Come Dancing (or Dancing with the Stars) I believe that the different dance styles can be said to be defined by heuristics. For example, the Tango has its own defining characteristics, which are very different to those of the Cha Cha Cha. Each Tango, however, can be very different when its interpretation is defined into an algorithm. I admit to being no dance expert, but assuming the metaphor holds, however loosely, then its means that kanban may thought of as a dance style, and it is valid to talk about Kanban Style.
About the Author
Karl Scotland is a versatile software practitioner with over 20 years of experience covering development, project management, team leadership, coaching and training. For the last 10 years he has been successfully applying Agile methods, and most recently has been a pioneer and advocate of using Kanban Systems for software development. Currently an Agile Coach with Rally Software in the UK, Karl is a founding member of the Lean Systems Society and the Limited WIP Society, and has previously championed Agile and Lean Thinking with the BBC, Yahoo! and EMC Consulting. Karl writes about his latest ideas on his blog.