Why Do We Need Self-Organising Teams?

“The best architectures, requirements, and designs emerge from self-organising teams”, the Agile Manifesto announces. This raises a few questions: What are self-organising teams? Why do we need them? What difference do self-organising teams make? How can we support self-organisation? Could there be any way to help this special kind of teamwork to emerge?

Surprisingly, there is relatively little material on what self-organising teams are about and how to support them effectively. Organisational development consultant Sigi Kaltenecker and agile coach Peter Hundermark are writing a short book “Leading Self-Organising Teams” to be published by InfoQ later in 2014.

This is the second in a series of articles that will connect readers with the topic. The series began with “what are self-organising teams?”. This article focusses on “why do we need self-organising teams?”. The third and final article will be “what is Leading Self-Organising Teams all about?”.

Why Do We Need Self-Organising Teams?

From the 1980s onward, we experienced a tremendous amount of changes:

• political changes, such as the end of the Soviet Union and the block of Eastern European countries;
• societal changes such as intensified migration or higher educational levels in many countries;
• demographic changes as documented by higher life expectancy and decreasing birth rates in the Western hemisphere;
• ecological changes, mostly referred to as global warming and climate change
• technological changes, e.g. in medicine, biology or communication technology, giving birth to a new generation of “digital natives”;
• economic changes, from the tyranny of shareholder value and the rise of the so-called BRICS countries to the global financial crises in 2008.

All these changes bring with them new demands. Organisations have no longer been able to choose whether they want to respond to these demands or not. Change has become mandatory. Trying to hold onto the status quo is like trying to keep the leaves on trees in autumn. For an organisation to be successful, it must adequately deal with the risks and use the opportunities every change brings along. In other words, the organisation must keep up with, or ideally be ever so slightly ahead of, the current market demands. How inconvenient then that this market behaves unpredictably. That which is ‘top’ today can be a ‘flop’ tomorrow; yesterday’s success factor can become a burden overnight.

“Business agility” turns out to be the new mantra for the successful running of an organisation in the 21^st century. Improvement and innovation have long since become mandatory for any organisational unit. Available opportunities should be used, new possibilities discovered, competitive edges honed.

Self-organising agile teams seem to be kind of a miracle solution to many of these problems. They are said to:

• achieve better results
• deliver more business value
• collaborate more effectively than micro-managed teams
• learn faster
• work with more motivation and fun.
• be more rewarding

While many managers seem to be busy projecting their wishes onto self-organising teams, they are blanking out an essential blind spot: self-organisation is as much about the management as it is about the team. The need for more agility is also nurtured by the fact that traditional command-and-control management turned out to be dysfunctional. Stifling bureaucracy, suffocating control systems and the empty rituals of planning and performance management are just a few symptoms of this dysfunctionality.

According to current studies, such as The Shift Index from Deloitte Center for the Edge, only one in every five employees is fully engaged, 75% of all employees lack motivation and passion, and only 15% of all teams are able to realise their full potential. Besides, there is growing amount of “change fatigue” as to the fact that many change initiatives do not achieve the intended goals. Rather than commitment, these initiatives are more met with an attitude of “not again!” There are no comprehensive figures but various sample surveys point to a proportion of between 60% and 80% of projects ending in failure.

There is a variety of reasons for this depressing failure rate: lack of transparency, too many change initiatives in parallel, weak change agency, missing feedback loops and last but not least obsession with detailed project plans, pre-designed milestones and the prediction of clear cut outcomes. Unfortunately, all the turbulence around us make a mockery of our plans and predictions. As Meg Wheatley arouses us: “It’s time to realise that we will never cope with this new world using our old maps.”

Let’s examine a map depicting the organisational paradigm common in the last century with a modern view to better understand what the necessary change is about (Table 1).

Table 1: Paradigms of Organisations

The table summarises some of the key differences between mechanistic and systemic thinking as outlined by Russell Ackoff more than 25 years ago. Even though the table tends to polarise a bit too much, it outlines the systemic context of past and future-oriented leadership. The dominant organisational paradigms resonate with the basic values and principles of two very different models of how to manage and lead: functional vs. holistic set-up, linear cause and effect vs. complexity thinking, administration vs. continuous innovation, shareholder value vs. interests of all stakeholders; change as exception vs. change as key driver of any business.

Thus, the former administrator of standardised business processes is supposed to become an organisational designer for high-performing teams. The ability to set clear goals, establish modes of decision-making, and free up resources are also a part of this. The trouble is that the principles and values of the mechanistic paradigm are still pretty much in place. They still guide old-school management practices in many organisations—and, perhaps even worse, the educational concepts at universities. Despite all the new challenges around us, the traditional MBA is still seen as key asset to qualify a manager.

But is business administration really what is needed to deal with the current challenges? As Jeremy Hope and Robin Fraser, founders of the highly influential Beyond Budgeting Round Table [hyperlink], put it: “For most organizations today, their success factors have changed and their strategy is changing, but their management processes, leadership styles, and cultures are lagging behind.”

This raises the question of what a future-oriented leadership model can look like? What is needed to meet the current challenges? Why do we think that effectively leading self-organising teams is key to succeed in the 21st century? What values, skills and techniques are needed to support rather than hinder self-organisation? Over the course of the last decade we got a lot of answers to these questions. From a variety of modern literature and from our own consulting experience we observe some recurring themes:

• old-fashioned command and control gives way to a modern culture that respects self-control without losing sight of the organisation-wide need for coordination;
• new forms of network-oriented leadership appear alongside hierarchical management in order to use the available expertise optimally, last but not least in response to environmental dynamics;
• if managers still try to control both people and activity, constricting team member’s freedom and inhibiting local change, they only create the conditions that threaten the organisation’s survival.
• to focus on self-control is the only way to show respect and effectively exploit/capitalise the capabilities of well-trained knowledge workers;
• encouraging decentralised decision-making while keeping overview by applying visual management, establishing fast feedback loops and selected team performance metrics is a sure path to better alignment and intrinsic motivation of teams;
• centralised, “heroic management”, the role model of the one superior director or captain, gives way to a model of decentralised, post-heroic “leadership as a team sport” that explicitly builds on mutual relationships and fast feedback loops.

Far from being purely theoretical, our old maps are kind of the source code for dysfunctional behaviour—and the root cause of numerous organisational problems. On the one hand these maps create a high degree of demotivation. There is an increasing amount of turnover and burnout, often leading to the loss of key players, who are tired of fighting windmills. This results in an obvious gap between what companies must achieve and what people actually want to invest. No wonder that the average life expectancy of organisations is meanwhile below 20 years. On the other hand, managers are forced to acknowledge a fundamental paradox: that they are individually responsible for the behaviour of a complex social system they cannot control. Amidst a turbulent environment, management inevitably has to deal with an often overwhelming amount of uncertainty, unpredictability and risk.

Given the present amount of complexity no single person is able to capture let alone process it appropriately. Mental overload is inevitable. Best case a manager can build on certain probabilities, worst case her actions and decisions are purely random. No “management-by” method offers an escape from this fate, whether it pretends to be scientific or not. Managers have to accept the difficulty of controlling social systems. Rather than superior directors of their organisations, managers are more like the proverbial fly ruling on the trunk of an elephant. The fly is convinced that it is steering the elephant, the elephant does not mind, and it makes the ride more interesting.

Both external and internal factors underscore the need to change how we run our organisations. In order to become more agile we must transfer more power and authority to people closer to the customer. We have to trust them with information and give them time to think, learn and improve. At the same time structural costs must be slashed and bureaucracy reduced if not eliminated. Lean is the right keyword for this effort.

The only way to achieve these goals is to empower our teams. We have to allow them to use the full amount of their expertise, not just to execute their work but to monitor and control themselves, make their own decisions and even design their processes. This may be seen as a question of natural respect. As Drucker pointed out some 30 years ago, knowledge workers such as IT experts have to have autonomy. Our experience shows that effectively using is both a question of individual as well as team training and organisational change. Again, self-organisation does not happen overnight. Since the containers for this self-organisation are still restricted in various ways, let alone chronically disturbed by micro-management and lacking work design, to capitalise on self-organising processes needs fundamental changes. If effective empowerment can be seen as an equation of freedom multiplied with capability, we need both to learn new things and unlearn old patterns.

This equation reminds us that self-organising is not a technical process. Although we have to deal with a lot of structural issues, there are always emotions involved: positive ones such as pride, excitement or fun, but also negative emotions such as confusion, uncertainty or fear. Both categories of emotions are two sides of the same coin and typical phenomenons of change processes.

From this point of view, it comes as no surprise that In most cases both management and team feel a measure of ambivalence when it is about to transfer authority. As always, when we question things people build their self-esteem on (e.g. roles, responsibilities, resources) some may feel overwhelmed while others are puzzled. As change management pioneers Doppler and Lauterburg show, there are three basic questions that immediately pop up when it comes to change in the first place:

Do I need to do this? Do I understand why we need self-organising teams? Are these teams mandatory or are there any alternatives? What do we expect of self-organisation?

Can I do this? Am I able to deal with what self-organising will mean to us? Do I have all the skills I need to become self-organising? What are my chances for good results? What counts as success under the new conditions?

Do I want it? Is self-organising interesting? What’s in it for me? Is there any risk of losing something: money, relationships, career prospects? Can I expect to gain something from the change?

“We’re all for improvement, but why do we have to change?”, recently a member of an operations team pointed out the ambivalence many feel towards self-organising processes. These processes cannot be simply imposed, professional facilitation and change management is needed from the very beginning:

• profound information—why self-organisation?
• clear expectations—how to measure the success?
• professional facilitation—how to guide the transformation?
• training and coaching—what do you need to know and do?

Conclusion

We have observed that change is the only constant in our world and “business agility” is demanded. Our old maps for running organisations are no longer valid; we need new ones based on systemic thinking. The devolution of power and granting of autonomy to the knowledge worker is essential to regain and retain their engagement. Self-organising teams guided by coach-leaders are central to the new operating system.

If we agree then that self-organising teams are something we both need and want, the consequent challenge is to discover what kind of new leadership skills are demanded to enable this self-organisation to take place, and how aspirant leaders might acquire these. We will explore this in the third and final article in this series “what is Leading Self-organising Teams all about?”

References

1. Ackoff, Russell L. (1986). Management in Small Doses. Wiley.
2. Beyond Budgeting Round Table | Beyond Budgeting Institute (2013).
3. Deloitte Center for the Edge | The Shift Index (2013). 
4. Denning, Stephen (2010). The Leader´s Guide to Radical Management: Re-inventing the Workplace for the 21st Century. Jossey-Bass.
5. Doppler, Klaus & Lauterburg, Christoph (9th Edition, 2000). Change Management: Den Unternehmenswandel gestalten. Campus Verlag. [in German].
6. Drucker, Peter (2001). Management Challenges for the 21st Century. HarperBusiness.
7. Hackman, J. Richard (2006). Leading Teams: Setting the Stage for Great Performances. Harvard Business Press.
8. Hope, Jeremy, Robin Fraser (2003). Beyond Budgeting: How Managers can Break Free from the Annual Performance Trap. Harvard Business Review.
9. Katzenbach, Jon R. and Smith, Douglas K. (2002). The Wisdom of Teams: Creating the High-Performing Organization. Collins.
10. Wheatley, Margaret J. (2006). Leadership and the New Science. Berrett-Koehler.

About the Authors

Sigi Kaltenecker is the joint managing director of Loop Consultancy in Vienna, helping individuals, groups and organisations to successfully master their professional challenges. He is a Certified Scrum Master, Kanban Coaching Professional and co-editor of PAM. Sigi co-authored the book “Kanban in IT: Creating a culture of continuous improvement” which will be published in English in 2015. Reach him at @sigikaltenecker.

Peter Hundermark is a Certified Scrum Coach and Trainer and a Kanban coach at Scrum Sense. He focuses on organisational development, change management and leadership development to help bring agility to the world of work. He is the author of Do Better Scrum. Reach him at @peterhundermark.