Downscaling SAFe

Whenever one starts to talk about scaling up an organization, it is important to define the areas of growth. What causes the organization to grow, where is it growing, where does it need to grow, how - and why?

In the case of Seamless Payments - a Swedish company behind a mobile payment solution called SEQR - three definitions of growth in two years’ time could be identified.

• First of all, the number of employees in the company has grown from 50 to 200.
• Second of all, at the same time the number of countries in which the product is available has increased from 1 to 12 (USA, UK, part of Eurozone, Sweden and Romania).
• Third of all, the potential number of consumers which could use the product has grown from 10 millions to approximately 600 millions.

Consequently, the challenges which the company faced were not only technical (how to deal with: multiple environments, feature requests coming from different markets, synchronizing work between teams?) but also organizational (how to deal with inevitable change of culture in organization which grew 4 times in 2 years?). The Scaled Agile Framework (SAFe) with custom modifications to it in accordance with Agile and Lean values helped the company to go through this period and prepare for further growth. This article describes the change that was done using a slimmed down version of SAFe that still maintained its core ideas. However, one needs to remember that the change was also possible due to major technical investments in the deployment pipeline, a Lean Startup way of thinking about building products etc., which are beyond the scope of this article.

To better understand the consequences of the change one should look at this from two perspectives: marketing and management. Geoffrey Moore in his book “Crossing the chasm” describes how companies with disruptive innovations (which SEQR undeniably is) may fail at early startup phase. The majority of such companies, according to the research, fails at transforming from early adopters of a product to early majority. Although mobile payments are not in early majority phase globally yet, Seamless is at the moment flying over the chasm by being already present in 12 countries. On the other hand, from a management point of view it is worth understanding Greiner’s growth model. With a fourfold increase of the number of employees the company was growing through direction. However, new markets and new features became so numerous that the employees started to experience a lack of situational awareness and purpose which resulted in frustration. This could be interpreted as an autonomy crisis in the aforementioned model for which a solution had to be found.

Implementation of SAFe at Seamless

The Scaled Agile Framework - SAFe for short - adds a business framework to support agile teams. It addresses the inherent shortcomings of scaling Scrum to large corporations by introducing tools and structure for creating a WIP-limited pull system from prioritized pipelines of high-level requirements to rolling-wave planned release trains.

SAFe is not only a way to scale agile organizations, but it also offers an interesting perspective on how to do it. In particular, it does away with the often chaotic and sub-optimized multi-project environment and creates clear and simple top-level backlogs from which agile teams produce a sprint plan and a slightly longer forecast. In theory, this should give teams a better heads-up on upcoming work, thus avoiding the dreaded "one-sprint planning horizon" often prevalent in agile organizations, and also give the teams their much needed focus and time to do the work properly.

But SAFe was created with the primary intent of introducing agile and lean thinking into big corporations. How can it be applied to a fast-moving small company that wants to introduce structure without unnecessary bureaucracy?

The core principles of SAFe are present in this modified version:

• The planning horizon covers about three months, being re-planned every sprint planning.
• A tactical, or program, level is introduced where teams show off how they coordinate their work and what their focus is in the upcoming sprints
• A strategic, or portfolio level is added where features and epics are funneled into one or more backlogs, which are managed by product managers and product owners.
• Finally, a technical/infrastructure backlog is created on the top level, managed by the architect who ensures that technical investments are motivated from business features, with business cases if needed.

From previous experience with small organizations, this system does indeed apply to their reality, reducing chaos and waste quite a bit, offering a good tradeoff between reduced waste and a slight increase in administrative overhead. For various reasons, the visualization of the top and mid tier of SAFe was implemented using a set of slightly different tools than prescribed by SAFe, which will be described next.

The backlogs of features, epics and stories were managed in Confluence and Jira, which proved sufficient for distributed teams to pull work from. A blueprint was prepared in Confluence to make it easy to describe ideas, needs and features in a way that could be understood at a glance, with further details available at demand. It was also required to describe early how the benefits of the realized blueprint would be realized and which decisions and lessons learned should be promoted to future releases. An added bonus with Confluence proved to be the discussion threads, often increasing the involvement from distributed experts without the need of extra meetings, meaning that the blueprint could start from an idea scribbled down on paper, or interview notes with users, growing over time into a business case complete with links to user stories, acceptance test criteria, design documentation etc. providing full traceability in a simple format once new features are launched

The overall WIP was managed using a Capacity Matrix. This is a visual board that lists all line functions providing capacity on one axis and projects/assignments/other work withdrawing capacity on the other. Whenever an intersection between a function and ongoing work was being over-utilized, a red signal would alert management that decisions have to be made to help the teams to move forward. Over time, a mindset change started to occur on the managerial level. While management had the clear goal to help teams and not only express their expectations it is also expected by teams to find a solution to a problem on their own (possibly aided by a ScrumMaster) before announcing a need for help. In essence, a red signal means that the teams have exhausted their options and will miss the next goal unless management support is provided.

Capacity Matrix in Google Spreadsheet

By monitoring the capacity matrix, which is updated weekly by team leads and product owners, managers can focus on the right decisions at the right time such as reassigning priorities or cutting scope. This concept does scale well because management can focus on strategic decisions and get involved into operational activities only when it is needed and also avoids the temptation of moving people between projects and tasks. Furthermore, by using this kind of visual planning micro management is avoided as the status and ongoing efforts are always visible in a format that can be understood by everyone. But before even reaching status red in the Matrix, teams have already tried to solve the issue through their primary planning tool - the Tetris board.

Tetris Board in Google Spreadsheet

The Tetris Board - no connection to a popular game with the same name - consists of a number of swim lanes - one per team - divided by time markers in the form of sprint boundaries. Here, teams illustrate through the use of colored sticky notes of different shapes which release or epic they will be working on in a particular sprint, and dependencies between teams are illustrated by allowing the colored notes to "overflow" swimlanes, occasionally resembling “Tetris” blocks. In Scrum terminology, the current sprint represents a commitment and the rest of the board displays the teams combined forecast on how and when they will focus on different types of work. For this case, it was also decided to color-code the type of work, meaning that epics, architecture / infrastructure, risk management and spikes all got individual colors, making it possible for teams and managers to assess the “mix” of upcoming work at a glance.

At Seamless, due to the distributed nature of the company (the Software Engineering department being present in 4 countries) the Tetris Board was first replicated by managers and ScrumMasters on physical boards, one copy at each location. After six months, the Tetris Board was moved online into a Google Spreadsheet document which can be accessed by anyone in the company. Having the physical boards for such a long time with duplicated information did cause some initial complaints over extra work and “old-school tools”, but when the boards were finally moved to a digital form, it was concluded from all sites that the purpose and joint ownership of the boards had not been made as clear had they not been present in a physical version to begin with.

What are the important points of this planning board? As mentioned before, the Tetris Board is updated directly by teams. Thanks to this they gain situational awareness and an understanding of the common goals - lack of these were one of the most significant problems during the fast growth of the company. In addition, people in the teams can, due to their collective knowledge and experience, identify dependencies more precisely between epics being implemented to avoid problems in future. This kind of ability is hard to find in a single person such as a Product Owner, Product Manager or Architect, although this is often expected by an organization and can be wrongly deduced from the agile principles.

Another very useful thing is that the Tetris Board provides a quick way of assessing the consequences of adding unplanned work, or stories going on far longer than expected. Teams and team members can easily see the critical chains of their work, adding buffers and managing risk accordingly.

Scrum and ScrumMasters in SAFe

SAFe uses Scrum on the team tier of the organization. As Seamless had tried Scrum on team level for some time, the concept of working in sprints with planning sessions, synchronization meetings, review and retrospectives were already in place. The role of ScrumMasters is toned down in SAFe compared to Scrum, and implementation of the framework is largely left to management. Coming from a previous good experience of having ScrumMasters act as agile coaches for the entire organization, an approach with dedicated ScrumMasters being authorized to coach the implementation of Scrum on team level was introduced, but the role was also further extended to facilitate the rolling Tetris planning and review. Interviews were conducted with the current ScrumMasters who for the most part were enthusiastic to the proposition of working as full-time coaches. In the end, only one of the current ScrumMasters opted for a technical career instead of a full-time coach position. (In practice, some ScrumMasters did help out with technical issues when needed, without compromising their primary job, but this was never required of them)

Other roles

To facilitate Capacity meetings and Tetris reviews (where the Tetris board is discussed weekly by stakeholders) the role of Program Manager was created. This role ensures that information is kept updated by teams and individuals and that the meetings are kept short and focused. The role of Release Train Engineer as described by SAFe was not used, responsibilities instead being shared between the Program Manager and a System Team (aptly named: “Rollout Team”) headed by a skilled test- and integration engineer. As was the case at Seamless, this was implemented as a trade-off, as it may seem suboptimal to the reader. There are actually two handovers which take place: at first, teams pass their work to the Rollout Team for testing on staging and, subsequently, they pass it on to Operations team to have it deployed onto the production environment. Problems connected with loss of information and context and little feedback from production environment back to developers may arise, something the DevOps approach tries to address. However, the transformation to a DevOps organization requires a continued investment in further cultural change and automation, which is beyond the scope of this article.

To raise the level of technical and integration awareness in teams, the role of Tech Lead was created. As an informal leader, the Tech Lead is responsible for ensuring that teams will pull relevant stories from the architecture runway and Tech Leads bring integration and architecture issues and design suggestions back from the team to the runway. To ensure that learning goes both ways, even when time is short, architects would regularly pull the tech leads together for discussion and learning.

From Projects to Release Trains

It is common for many product development organizations to be stuck in the traditional model of project budgeting and execution. Capacity/resource management is often performed ad hoc, projects being funded when needed, and project managers often rewarded for delivering results on time and budget, without concern for other ongoing work. As we know from W. Edwards Deming, optimizing a single workflow, project or business leads to sub-optimizing the whole system. This very often means that in reality people are pulled between work, which is re-prioritized without much afterthought, leading to a lot of time (and money) being wasted in context-switching, also leading to increased costs from poor release quality, poor design, and increased variability in forecasts.

Introducing a WIP-limited program execution where work is planned in release trains and budgeted according to the number of needed, and possible, parallel trains, brings an end to the costly fire-fighting individual scenario.

The transformation is often contested and difficult in practice, something that also proved true in this case. Initially, project managers opposed the idea, advocating the need for projects to control the introduction of new markets and products. The strategy used was to reduce the focus of a project to all activities surrounding a market launch or a new product launch (including engagement from Marketing and Sales team), providing input to and getting synchronization and forecasts from release trains. This revised scope was initially adopted by Project Managers, and later taken over by Product Owners in order to transfer execution to roles with a lifecycle view of the products and markets.

As far as budgeting is concerned, Product Management (consisting of a Head of Product Management and Product Owners) is responsible for this. They collect requests from all the departments in the company and create a prioritized backlog from which teams can pull work. No specific formula can be given in this article for how important a given epic is. However, things taken into account are: deadlines for entering new markets, CEO’s vision, investor’s expectations, ROI (in which income, attracting new consumers and increasing loyalty of existing consumers play major roles) and effort estimation from the teams. It is worth emphasizing that a partnership of trust between Product Management and Software Engineering has been built which results in scope modification discussions whenever the estimates provided by the teams are beyond a planned budget. It is not uncommon in other companies that engineering becomes a servant to a specific business unit and delivers what is ordered. At Seamless however, mainly due to the aforementioned partnership, empowerment of teams and pull system (obtained via the mechanics of Tetris Board and Capacity Matrix) the Software Engineering department takes an active part in driving the business of the company.

As far as the Release Train is concerned, it brought a necessary cadence to the way the organization works. It was vital during the initial chaos that was caused by the fast growth of the company. However, it is an intermediary step rather than ultimate goal of delivering new features. The ultimate goal is to deliver as frequently as possible so that any release ceremonies (like rollout planning meetings) become pointless as they would have to take place all the time becoming a nuisance eventually. That can be obtained using technical investments stemming from DevOps culture in the organization. Significant amount of attention is being paid at the moment to that change.

Conclusions

The transformation which happened at Seamless can be summarized with one sentence: scaling up is about utilization of knowledge and experience of teams because they deal with the scaling consequences on a daily basis and very often understand it much better than management. However, management responsibility is to create a system which supports this setup, empowers teams and influences the appropriate organization culture.

The system for teams is obtained using tools like the Tetris Board, Capacity Matrix and Release Trains which provide sufficient level of visualization for the whole organization to perform the planning and forecast the future, understand complex dependencies and being able to find an optimal WIP limit not only for teams but most importantly for the whole company. ScrumMasters as Agile Coaches become change agents in the organization and should ensure that the vision is clear on all levels.

As far as empowerment of the teams is concerned, it is obtained by a pull system in which teams pull features to work on from backlog created by Product Management. It is the teams which influence the order of things to be done due to collaborative work on Tetris Board. Meetings on which the Tetris Board is discussed between teams and Product Management (held once a week for half an hour) provide an opportunity to discuss priorities, sync in terms of goals and vision and increase situational awareness. This, in turn, gives teams the necessary context for making better and more conscious decisions every day. Moreover, the mechanics behind the Capacity Matrix put emphasis on teams having the freedom to make decisions and, what is most important, take responsibility for it. On the other hand, it prevents management from falling for the temptation of micromanaging their work and lets them focus on giving enough support for teams to do their job the best they can.

The last factor is nurturing a proper culture in an organization which supports the system and empowerment of teams. There are no easy recipes for this apart from the ideas and principles one should follow. Those ideas are: trust, respect, a common goal and partnership between different departments to achieve this goal. That is why it is so crucial that change agents are mostly recruited from ScrumMasters since they have the inherent willingness and enthusiasm for transforming an organization and helping it improve at all times.

To sum up, the case study of Seamless can be an evidence that small or medium-sized companies can benefit from a scaled agile framework with custom modifications. Moreover, if the company succeeds and grows even more it will, hopefully, be ready to absorb the turbulences connected with it. The future will verify this assumption. However counterintuitive it may seem, the response to scaling up an organization should be empowering teams and creating supportive system for them rather than creating a multi-layer management structure in which a lot depends on single people who very often become overwhelmed in trying to grasp the complexity of the world they work in.

About the Authors

Mikael Lundgren started using Scrum and Lean Product Development in 2001, after experiencing six years of increasing frustration with the contemporary means of directing work in the software industry. A former software developer, project- and development manager, Mikael became a Certified Scrum Trainer in 2006, and later founded Levla AB as a platform to help entrepreneurs and leaders create competitive, creative and sustainable organizations. Examples of companies that have worked together with Mikael in the past to transform their organization include Spotify, bwin, ABB and Assa ABLOY. As an author of books on agile product management and the ScrumMaster profession, Mikael is also a popular lecturer and speaker. He holds an MSc in Computer Science from Uppsala University. Visit Levla and Leanspired to learn more, and follow Mikael on Twitter (@leanspired).

Tomek Pająk (Lodz, Poland) went through several roles in IT: Software Engineer, IT Architect and Engineering Team Lead. At the moment he is Software Engineering Manager at Seamless Payments responsible for services built on top of the core product SEQR - the mobile payments solution available in 12 countries (including USA, UK, Sweden and most of the Eurozone). To obtain competitive advantage of the products he utilizes Lean Startup and Scrum (for product development) and DevOps (for strong IT performance). At the same time he is Coach at Sages helping other companies to improve their businesses by adoption of agile/lean concepts and certain technologies. Tomek received his MBA title from Akademia Leona Kozminskiego and MSc in Telecommunications and Computer Science from Technical University of Lodz. He speaks at international conferences: Agile Lean Europe, Agile Eastern Europe, Atmosphere, Agile By Example. You can reach him at LinkedIn and twitter (@tomekatwork).