Coping with Change on Scrum Projects

Abstract

Adopting an Agile approach to software development requires much change in an organization, whether it is corporate culture, individual roles or processes. As an organization switching to Agile, one will have to learn to cope with such change.

In this article, I refer to Agile, Scrum and Lean. So, it is prudent that I preface this article with a definition of what I mean by the term "Agile",". In my opinion, Agile is the umbrella term for an increasing number of new project management methodologies all centered on iterative, incremental software development. To name just a few popular ones, Lean, Crystal, Scrum, DSDM and XP come to mind.

Scrum and XP, and in particular the two in combination, are by far the most popular Agile methodologies being practiced today. Although Scrum does not include in its definition anything related to engineering discipline, it is impossible to separate the two. So if you ask me what I mean by Agile, let me define it here:

Agile = Scrum + XP

Recently a number of thought leaders have successfully applied Lean thinking to software project management and it would be foolish on my part not to make mention of it in this article.

So it is in this context that this article is written.

This article surveys the expected variation of different roles in the Agile organization and proposes techniques with which to better handle the transition to Agile methodologies from traditional Waterfall. The following roles are discussed in this article: Customers/Stakeholders, Product Management, General Management, Project Management, Developers and Quality Assurance. Although I do provide more detail on technical roles, largely based on my own personal experiences being more technical.

The Agile approach is centered on efficiency in realizing short-term project objectives. Contrary to the waterfall model, for example, functional pieces of software are produced at 2-4 week intervals. Such a system ensures both team motivation and client satisfaction through frequent inspect and adapt cycles. Agile's ability to recast individuals in positions and in settings that maximize efficiency, individual potential and output, yields a software development strategy in which associated changes are justified by its results: an economical and result-oriented approach to software development.

Introduction

Most people do not like change. I know I don't. But one thing is certain, adopting an Agile approach to software development requires considerable change in an organization, whether it is corporate culture, roles or processes. As an organization switching to Agile, you are going to have to learn how to cope with change. This article deals with what teams new to Agile can expect and how to better adapt to this new and invigorating environment. This article covers the changes one can expect for the following roles in an Agile organization:

1. Customers/Stakeholders
2. Product Management
3. General Management
4. Project Management
5. Developers
6. Quality Assurance

It is important to note that not all the information provided will apply to every organization as each team operates differently and possesses different skill-sets. The following changes are reflective of some of the experiences I have encountered throughout my time managing development projects as a ScrumMaster. Here are some of my findings:

Customers/Stakeholders

With traditional waterfall projects, customers are typically held at arm's length and only involved at the beginning and end of a project. Nine times out of ten, we (development teams) force customers to be candid and up-front. We make it quite clear to the customer that any change after the start of a project requires change control processes and penalties. Then after we have finished coding, which can be many months after commencement of the project, we hand the code over only to find out that "we got it wrong." Needless to say, this approach is a recipe for disaster.

So on Agile projects, customers need to be far more engaged throughout the development process. In fact, on Agile projects change is expected and customer feedback is welcomed.

1. Customers are expected to participate and provide input at all "inspect and adapt" points. This minimizes risk and provides customers and stakeholders with options. Note: On XP projects Customers are required to be part of the team.
2. Customers should work with the Product Owners to define user stories and to elaborate on the user stories during planning meetings.
3. Customers should work with the Product Owner to prioritize the Backlog.
4. Customers and stakeholders participate in the end of Sprint demos and, depending on the relationship between customer and Product Owner, the Customer can even participate in the Sprint retrospective.

Product Management

If you are a Product Manager on a team shifting to Agile, you too can expect change. For starters, your title changes from Product Manager to Product Owner. But that's not all. Product Managers in traditional environments are mostly front and back-end loaded. Besides their specific marketing duties, Product Managers are typically responsible for the up-front Product Requirements Document. On the other hand, Product Owners are the proxy for the customer. Shifting from a Product Manager to Product Owner on an Agile project requires the following changes in duties and responsibilities:

1. The Product Owner is an active participant on the team. The buck stops squarely with the Product Owner as they become ultimately responsible for the product direction and priorities for a project.
2. A Product Owner should be present at the Sprint planning meeting. The Product Owner generally owns the first part of the Sprint planning meeting and is responsible for defining the goals for the Sprint as well as elaborating on the functionality of each user story planned for that Sprint.
3. The Product Owner is responsible for creating, maintaining, and prioritizing the product backlog.
4. Contrary to popular belief, the Product Owner is also required to avail him/herself to the team at all times. Whether or not the Product Owner is present at every single daily stand-up is debatable, but the more engaged the Product Owner is through out, the more chance there is of achieving success.
5. The Product Owner also becomes partly responsible for defining the acceptance test criteria and, in this regard, are somewhat responsible for the quality of the product output - especially since such acceptance test criteria allows the development team to build in quality from the start.

Therefore, as a Product Owner, expect to be more involved day-to-day and be prepared to get your hands dirty.

General Management

Since Agile teams are supposed to be self managing, where does this leave general management and what is its role? Hopefully the following will give you some food for thought.

1. Managers should be available to act as a sounding board for the team, especially if the Manager has the appropriate experience - there is still no match for experience. This is why Toyota has the Chief Engineer as it's Product Owner. This support can help teams avoid mistakes and therefore save the team considerable time.
2. Managers should be present to help with the heavy lifting i.e. they should be assisting the ScrumMaster to eliminate "BIG" blockers or impediments that require management involvement i.e. could require budget approval to move forward.
3. New teams especially require support and buy-in from executive management in order for an Agile process to take hold. Managers must go to bat for the team to keep them on an Agile track. It's so easy for teams to slip back into their old habits.
4. Managers are expected to act as "coach", assisting team members with career path planning, carrying out performance reviews, training or organizing training.
5. Managers should focus on the longer-term strategic initiatives in the company (i.e. figuring out how to deal with competitive threats, increasing sales, reducing costs etc.).
6. Based on the outcome of the strategic initiatives, managers should work closely with the Product Owners to define and prioritize the longer term "Epics" and to create the long term product road maps

Project Management

Scrum essentially does away with the traditional "Project Management" role. So where does that leave the employees? Most often, Project Managers transition into the role of the ScrumMaster quite naturally. However, believe it or not, great ScrumMasters also come from Development and QA. Trust me, the ScrumMaster role is not even vaguely close to that of the Project Manager. Therefore, it's important that the ScrumMaster candidate acquire some training. Scrum Certification, although not a must, is highly beneficial. So what can you expect as you transition into this role?

1. To start, as ScrumMaster you must come to terms with the fact that the team owns the schedule, so no more updating the good old MS Project Gantts any longer. Instead, you have to sit back and maybe initially prod the team to keep their daily estimates current.
2. As ScrumMaster you have to ensure that the Scrum process is followed, that everyone understands Scrum and what is expected.
3. You will be expected to remove impediments, or assist in removing impediments, as quickly as possible.
4. You must ensure the team stays on track by reminding the team of the Sprint goals at every meeting.
5. You will be organizing the daily Scrums (a daily meeting that answers what did you do yesterday, what will you do today and are there any impediments in your way), and ensuring that they are kept at the same time and place.
6. You must ensure that the subtleties of Scrum and, in particular, that the inspect and adapt points are taken seriously so that they are used in the manner for which they were designed. This allows the team to learn and improve Sprint by Sprint.

Let's now take a look at how the lives of Developers and QA Testers are expected to change on Agile (Scrum) projects. Frankly, I think that for these two disciplines, life gets much better as Developers and Testers benefit most from transitioning to Agile.

Developers

Developers love to do things right. Nine times out of ten, they'll argue against taking short cuts. They hate the fact that they are forced to deploy code that they know deep down is less than stellar. As with anything in life, making the shift to Agile swings the pendulum to the complete opposite side. So for the one out of ten code hackers out there, beware. The biggest change a developer can expect from a shift to Agile (Scrum + XP) is that engineering discipline, or rigor, should be set to the max. So what can you expect?

1. You no longer have to estimate tasks on your own. This activity, arguably one of the hardest, should be done as a team preferably playing planning poker. As a result, you'll get better at estimating and you'll take a much broader outlook on estimates. Additionally, you won't have to shoulder the complete responsibility for a bad estimate.
2. You must learn to pair program. Assuming you are implementing XP practices (highly recommended), you will be doing this more often than not. This is a huge mental shift that developers must make because for the first time, you are forced to open up your code to peer review. This means that you are being critiqued every step of the way. It's hard at first, but the benefit is huge; you are going to be a better programmer for it, guaranteed! You'll produce higher quality and more maintainable code.
3. No longer are you expected to throw code over the fence and expect testers to find the bugs for you. Agile is built on the premise of sustainable throughput, and in order to do this and keep the momentum of the development pipeline, it is imperative that quality is built in right from the start. So, you're going to have to learn how to write unit tests. You will get to learn lots about Test Driven Development. This is where your skills as an engineer are challenged to produce the best possible, highest quality code. The rewards are great, trust me on this.
4. You should participate in the complete end-to-end process. This will give you a new perspective on things, as you are a part of the initial user story workshops. Consequently you get to hear issues first hand from the Customer/Product Owner as they are there to elaborate at the Sprint planning meeting. You should also be a part of the retrospective at the end of the Sprint so you get to close the loop, reflect on what went right, what went wrong and help make improvements Sprint by Sprint.
5. You won't have to burn continuously. In fact, if you are true to Scrum, you shouldn't have to burn at all. Velocity tracking sets your maximum sustainable team throughput, which governs how much work-in-progress there is at any given time. Too much in the pipeline will result in delivery of less than expected. Our team on Agilebuddy (Agile project management software that I co-created to help development teams transition into Agile and manage their software development projects) deploys new builds to production every two weeks without fail. We accomplish this by being optimally primed for just enough work. We can learn a lot in this regard from Lean thinking and the Toyota Production System.
6. If you adhere to the process, you won't have to worry about death marches or code blues. Rather, at the end of each Sprint, you will feel proud of what you have produced and will never want to work any other way again. This requires that all Stakeholders be bought into this new process right from the start (a pre-requisite for a successful Agile implementation)
7. At the end of every Sprint, you have the opportunity to demo what you have done - this is the icing on the cake. This is a time for you to shine rather than hide, as opposed to waterfall projects where you would never know where or when the next bug might surface.
8. You will work as a team and cross each milestone as a team. This is not about I's and Me's. Instead, you will seek out opportunities to help team members when in need.

QA Testers

Having worked in a waterfall environment most of my career, I am all too familiar with the "death march." This inevitably leads to shrinkage in the time left for QA to test the software. Fortunately for Testers, as with Developers, this scenario is altered by the Agile method . How is this accomplished?

1. By participating from the beginning of every Sprint. You may wonder what value can a Tester add this early on in a project. However, because you must participate in the Sprint planning meetings, you will get to hear up-front what user stories are planned. In fact, you will be part of the process of elaborating the user stories themselves. You will, for example, help to define the user story acceptance test criteria. This significantly helps Developers understand what they need to do up front for the tests to pass. As a result, the quality of the code delivered, as well as the "accuracy" of code (i.e. how closely the code matches end user requirements) is dramatically improved.
2. You will be a part of the planning poker sessions to help size (or estimate) user stories. Participating in the estimating process means the company receives better overall estimates of user story complexity. This leads to more accurate estimates, better plans, less stress and happier teams with higher morale.
3. You will be involved in the creation of unit tests. This is one significant way in which Testers can enhance their skill sets. This is an area where testers can add a large amount of value to the process with their analytical and "how can I break things" mindset.
4. You should automate as many of the functional tests as possible. As a result, you will contribute to the overall effectiveness of the team and its productivity. Lean thinking suggests that teams need to be more concerned with building in quality and minimizing waste, as this has more to do with improving cycle time (from concept to cash) than any other factor. So, by writing automated tests, you are contributing to the overall quality of the codebase and as a result reducing overall cycle time.
5. You will be a respected member of the team rather than a second-class citizen with the right to "stop the line" (Toyota Production System). For example, any bug that is found even early in the project should be resolved as soon as it is found. The Agile mindset solves the traditional project management problems by building a cohesive team and treating each of the functional areas as development. In fact, Scrum only refers to Developers on a project (i.e. if you're a Tester on a Scrum project, you're just another Developer who has deliverables on the Sprint backlog).
6. Because Developers are writing unit tests, you will no longer be receiving extensively buggy code. So you can now focus on the more important aspects of testing edge and boundary conditions, as well as automating your unit tests.
7. Scrum/XP and Lean practices expect that all code delivered at the end of the Sprint is DONE (i.e. unit tested, functionally tested, integration tested). So you will have the time to get these completed each and every Sprint as all work (including testing) is accounted for in the Sprint plan - as opposed to just having, for example, the last 3 weeks of a project dedicated to testing.

Testing has certainly matured since Agile methodologies have taken hold, and testers are much more respected in the industry today than ever before. Testing is now essential to the development of good software as it ensures that the software does what it is meant for and performs as intended. With concepts such as TDD and BDD, the testing effort has shifted from a traditionally tail ended process to a front ended process, and as a result, quality is built in right from the start. I am convinced that this is single-handedly the number one reason for higher quality software, happier customers and motivated development teams.

Conclusion

Agile adapts the development setting so as to harness the greatest amount of working potential from a team and translates that into an incredibly efficient software development cycle. It also allows for the efficient and productive engagement of individuals in software development; eliminating corporate hierarchy in favor of a functional team. Transitioning to an Agile organization and adopting a Scrum approach requires a concerted effort and a willingness to change across all functions of a software development team. Initially, the change associated with the implementation of Scrum seems daunting. It is all too easy to be distracted by the technicalities of the shift, without focusing on the basis for this change: a restructuring of an approach to software development that is ultimately more efficient. Therefore, I have outlined a set of guidelines to aid you in the process. With the transition discussed above and an understanding of the mentality behind the move, the shift becomes more of an intuitive step in the formation of a productive software development company, rather than a voyage into uncharted and threatening territories. So whether you are new to Agile, in the process of transitioning to Agile, or have already adopted an Agile approach, I hope that this article has given you some direction in how to better cope with change on Scrum projects.

Jack Milunsky
As Chief Operating Officer and Scrum Master, Jack Milunksy heads software implementation at Brightspark Inc. He leads the teams’ efforts in building and implementing innovative products using the Agile methodology and scrum tactics. Jack is also the co-creator of Agilebuddy, Scrum project management software used to help development teams transition into Agile and manage their software development projects. He has lived and breathed Agile and Scrum for many years and received his Scrum Master certification from Ken Schwaber, the founder of Scrum.

Jack combines 18 years of experience managing software development teams both large and small. He is an early adopter of Agile and has a great passion for early stage startups.

Prior to joining Brightspark, Jack was VP of R&D at Tira Wireless where he was responsible for driving technological innovation in the company. He has held many senior positions in Hi Tech companies across diverse industries. He was the Director of R&D for Delano Technology Corp and he also served as the Director of R&D for Symantec/Delrina Corp.

Jack holds a BSC Elec (Eng) degree from the University of the Witwatersrand in South Africa and completed Business Administration from the Stanford University Business School.