Building Your Own Agile Team Maturity Assessment

Often, as a scrum master or agile coach, you probably feel like you have a pretty good idea of what it looks like for a team to mature. You understand the behaviors that should be demonstrated on a day-to-day basis in order for the team to maximize delivery of value. But how do you get the team to buy-in? How do you bring awareness and share knowledge on what this journey looks like? An agile maturity assessment just might be your answer.

An agile maturity assessment can help teams come to a common understanding of what agile maturity looks like and what steps they can take to get there. In this article we are going to dive into the value of assessing things like how the team has implemented agile principles and how technical practices are aligned with the outcomes you want to see. I will give you concrete examples you can use and will help you learn how to build an assessment for your teams and/or organization that is fit-for-purpose. When you are done reading this article, you will have everything you need to get started. Let’s dive in.

The Value of Measuring Team Agile Maturity

There are many benefits to assessing agile maturity on a team. It helps us to understand if a team is being agile or just doing agile. A team is doing agile when they are just going through the motions without understanding or buy-in of the agile principles and values. A team becomes agile when their mindset and behaviors shift to align with agile values and principles.

When a team is just doing agile, not much has changed, except for maybe they have more meetings on their calendar. They may still work in silos or limit their interactions with customers. You might see work being broken down into mini-water fall stories, with a design story, then a build story, and finally a test story. Prioritization is probably determined by project dates that are decided upon by those who are not doing the work. And worst of all, overall morale is still pretty low.

When a team becomes agile, you see a change in how they work together, how they think about the work, and how they approach things technically. They are truly customer-focused, and are always looking for ways to improve things. They create a learning environment and share knowledge with each other. They focus on delivering value in small increments each iteration. The team becomes empowered to use empiricism to predict delivery timelines; they fail fast and are not scared to be transparent about those failures and what they’ve learned. They strive for technical excellence and innovative thinking emerges. The team is happier and so are their customers.

To truly understand where the team is on the spectrum of doing agile and being agile, it is important for the team to self-reflect, be transparent about where they are and how they behave, and make plans to continuously increase their agility. By going through this assessment a team will learn things about themselves. Information emerges that they may not have been aware of and they gain a common understanding of where they want to go next.

Assessing Team Agile Maturity

There are a lot of great assessments on the market that have already been created for you. They come with in-depth metrics, technology, and reporting. The question then becomes buy vs. build. It would be nice to be able to use an assessment that has the measures your organization needs with all the built-in reporting. However, I have encountered a few challenges with this. First of all, they are pricey, and many times organizations don’t want to fund something like this until they see the value. Second, they require a lot of effort to get started; there is training on facilitation, software configuration, security issues to address, the list goes on. Unfortunately, I haven’t found one that is perfectly fit-for-purpose for all organizations, teams, or agile frameworks. Often there are so many dimensions, questions, and measures that it becomes overwhelming and simply unusable.

My recommendation is to build an assessment that covers the basics and gives you enough information to act. You don’t want to measure things you won’t use. KEEP IT SIMPLE and maximize the art of the work not done. Be intentional about what you include, because you may not want to include measures for things your organization is not ready to change. For example, if you know the structure of your teams is an issue, but leadership has vetoed any change in this regard multiple times, don’t measure it. You already know the team structure is an issue so assessing this will not provide value.

You don’t have to start from scratch either. There are many assessments that have been shared in agile communities you can use as your starting point. You could also start with the assessments and facilitation guide (linked below) I have created in collaboration with several other agile coaches. The most important thing is that it is fit-for-purpose and provides value to you, your team, and your organization. Take out the things you don’t need or that your organization isn’t ready to measure, and add the things that are missing. Collaborate with other scrum masters and agile coaches in your network to gain great insight on new topics you want to assess.

Full Assessment Examples and Facilitation Guide

As you are creating your maturity assessment, be very descriptive in what behaviors you would expect to see from your team at each level. The maturity levels should be behavior-driven and you should remove subjectivity as much as possible. For example, if we are measuring our maturity in visualizing work, you would start with the least agile behavior (the team is not visualizing their work), then graduate to the next reasonable level (the team is visualizing their work but it is in multiple tools), and then the next level (the team is visualizing all work in one place and has an up-to-date board), and so on. It is critical to approach it this way so that each level means the same thing for all teams and it is not open to interpretation. This will allow you to see trends across the organization as well and know the data is accurate.

When selecting the topics to assess, there are some fundamental things that will provide a lot of value. Assessing how teams are doing with some basic agile principles and practices can be very beneficial. Evaluating things like how they prioritize, how they collaborate with customers, how they visualize work, and how often they are meeting commitments can help identify places where the team has not adopted practices that are going to help shift their behaviors and experiences. 

Assessing Agile Principles and Practices

When assessing agile principles and practices, you are looking at those that are going to help your teams be successful. Refer back to the Agile Manifesto and select those items that you feel are absolutely necessary for your teams to be successful. Remember, while they are all important, you want the assessment to be lightweight so you may not want to assess all 12 principles. Consider things like prioritization, visualization, continuous improvement, customer collaboration and predictability as a starting point. You want to assess the adoption of agile principles that start to drive behaviors that shape a mindset shift. Here is an example of an assessment of a team's ability to prioritize.

You’ll notice that each level of maturity has a very specific description of a behavior that would be demonstrated on a team. It is pretty difficult to be subjective using this approach, so you will get consistent results across teams. This will also give you insight into trends across the organization so you can identify areas of opportunity that are wide-spread. It is important that teams can use the information gathered to improve, so the interpretation of the results is very important as well. For example, if your team scores a 1 on prioritization, it is an indication they are working on many unplanned items. You may be seeing symptoms of this in missed sprint commitments or overworked or burned out employees. The team might be busy but they are not necessarily delivering value. The assessment helps you and the team to understand the cause of some of the symptoms so they can start to introduce change that would help manage unplanned work. This way, they can ensure they are working on the right thing at the right time so they can meet sprint commitments and work at a sustainable pace.

Here is an example for assessing Predictability.

Here again, you can see each level of maturity has a very specific description of a behavior that the team might demonstrate. For example, if your team scores a 2 here, it would indicate they are getting good at understanding the amount of work they can bring into a sprint, they are estimating accurately for each sprint, but they are not yet looking to the future. They are not using what they know about their velocity to help customers understand when something might be delivered several sprints down the road. This might be an opportunity for the team to introduce story mapping or bucket estimation to help them set expectations more effectively with their stakeholders.

Assessing Team Dynamics

Next let’s take a look at how you might assess team dynamics. When we think about how a mature agile team might work together, we expect to see things like swarming, knowledge sharing, trust, and accountability. These are all things we can measure through an agile maturity assessment. Some topics you may also want to consider assessing are Learning, Conflict Management, Collaboration, Team-Centricity, and Trust.

Here is an example of an assessment of teamwork.

In this example, we are looking at how the team sees themselves. Do they feel they are a team or a group of individuals? Are they accountable to each other or are they working alone? Assessing this will help the team to identify areas where they can work more closely as a team, share goals, and collaborate. Just the art of discussing this can increase transparency and trust. It can also help you, a Scrum master, or agile coach to understand where individuals are at so you can work to mature the team through one-on-one coaching opportunities or through facilitation of team building activities.

Here is an example of how a team might assess the level of trust across the team.

The behaviors here have been customized to a particular organization that was having issues with people being comfortable sharing knowledge and allowing others to do things they were used to doing. You can customize the levels of maturity to meet your needs and give teams the information needed to act. For example, if you have a culture where lack of trust shows up as leaving people out of decisions, you might write your behavior descriptions differently. Level 0 may state “Individuals make decisions in a silo and do not consult others on the team, and Level 1 may be “Individuals sometimes make decisions in silos but often collaborate to ensure the team is all in agreement” and so on.

It is really beneficial to understand how the team is working together and how aligned they are with agile principles, but we’ve really only addressed process and behavior up until this point. We are missing a critical piece of the puzzle, and that is how the team approaches things technically. If the team is missing this piece they are really going to struggle to deliver value. Again this can be customized based on where your team is at with different technical aspects. Some categories you might consider assessing are; Quality, Testing, Technical Debt, Definition of Done, Code Deployments, and Simplicity. If your teams are more mature in their technical approach, you may want to consider assessments for CI/CD implementation, data science, AI (artificial intelligence) implementation, or other emerging technologies.

Here is an example of a technical debt assessment.

Here what we are looking for is how often the team implements changes that introduce debt or code that needs to be cleaned up into the environment. A mature team has technical debt documented and is prioritizing the work to remove it from the environment, knowing that it will slow them down and impact their ability to deliver value. They would also not be introducing more debt into the environment unless it was intentional and they immediately built a plan to resolve it. Sometimes technical debt is unavoidable but a mature team always has a plan to resolve it in the near future. They pay their debt back quickly so it doesn’t slow them down.

It can also be helpful to understand what the team norm is around code check-ins.

Daily commits are extremely important because they are easier to review and easier to reverse, which reduces risk. Understanding how the team approaches this can help drive conversations and increase understanding of the value of frequent code check-ins, which can help the team start to shift their behaviors. There are many other technical aspects you could include in the assessment; just make sure to only include those you and the team will and can act on and keep it simple. For example, if DevOps is not yet part of the culture, you may want to introduce this iteratively and only begin assessing once teams have implemented the practices.

Assessing Organizational Agility

The team can only be as successful as their environment allows. If the way the teams are working is at odds with the culture of the organization, their ability to effectively deliver value and mature in their agility will be stifled. Some topics that are useful to evaluate are: leadership Support, prg structure, customer support, business value, and project management techniques. You can assess each team's perspective on these topics and utilize the data as a way to coach up and influence cultural and organizational agility. It is important to set context here, as the team’s exposure may be limited. Define what they should consider when responding. For example, you may only want them to consider their direct leadership’s behaviors or the structural composition of their team alone, not others they know about in the organization.

Here is an example of how you might assess leadership support.

Again this is from the team’s perspective and you may need to define what you mean by leadership to get accurate data. If the team scored this question as a 1, you might take it as an opportunity to share the data with your leadership group, coach them on the agile values and principles, and help them understand how they can demonstrate support for the teams and become agile leaders. I often use individual counts across all teams and then present it as X% selected level 1, x% selected level 2, etc. along with any anonymous comments. This protects the safety of the teams and ensures you get real feedback around sensitive topics.

Here is an example of assessing organizational structure.

Do you really have long-lived, dedicated teams, or is it a façade? Are members of the team still dedicated to projects, or does the work flow through one backlog that the team organizes around? Even if you know the answer to this, having the data from the assessment can speak very loudly when working to influence leadership in this area.

Facilitation Techniques

I guarantee you most of the information you need to build an assessment that will work for your organization is already in your head. I encourage you to identify the key areas you think need to be assessed, describe what maturing through each category would look like and facilitate it as an assessment with teams in your organization. However, I want to also give you some facilitation techniques because how you collect the data will influence the outcomes and the quality of your data.

• Use a neutral facilitator – it can be helpful to get another scrum master or agile coach to facilitate the assessment.
• The team should self-assess and not be influenced in determining where they are at on the maturity scale.
• Use a voting technique similar to planning poker, having each team member select the level they feel the team is at and all showing their vote at the same time.
• Work to consensus, not majority – you will need to discuss until the entire team agrees. This will need to be time-boxed so if you cannot facilitate the team to consensus, table the topic and come back to it. Sometimes the team will not fully agree but as long as they do not strongly disagree you can select a number and move on.
• Create a safe, open environment – no management should be present during the assessment. Results for specific teams are not shared outside the team, only trends.
• There are no bad results, just information that can be used for continuous improvement.
• Limit focus for improvement: select 1 – 2 areas to focus on and set SMART goals.
• Re-assess at least every six months to trend results and gather information on how the teams and organization are progressing against their goals. Keep the goals visible, possibly as a retro agenda item or backlog story.

Maybe you are thinking this is all covered as part of your retrospective process. While a retrospective is a great forum for this conversation, traditional retrospectives don’t typically address the topics you would discuss as part of an agile maturity assessment. One or two topics that are covered in an assessment may emerge as opportunities identified as a team, but it is not a comprehensive review of all items in an assessment because it is focused only on the last sprint. This makes a formal agile maturity assessment a critical piece to the success of a team.

My Learnings from Team Maturity Assessments

Team maturity assessments have given me and the teams I’ve worked with so much insight into where they are on the spectrum of being agile vs doing agile. They have helped me to see trends across the organization as well as cultural impediments. The data collected has helped me as a coach to understand when to partner more closely with the PMO to update processes, identify subcultures where different values are in place, as well as identify training gaps. There are so many benefits and things you can learn from as assessment. Whether you are focused on one team or the whole organization, an assessment is key to bringing transparency and facilitating a continuous improvement plan.

I once worked with a group to assess all teams, both Kanban and Scrum. Through trending the data, we were able to identify that the Kanban teams had not implemented any Kanban practices (with the exception of creating Kanban boards). We were able to identify a systemic issue with the rollout of the framework for those teams and were able to then address it through training and coaching.

I have also learned that conversation is key, so do not just send out a survey. Teams learn so much when they discuss the topics and behaviors. They also get a chance to ask questions, increase their understanding of agile maturity, and give you feedback on how you can make the assessment better. Trending the data every six months gives great insight into systemic issues.

Finally, I learned very quickly to keep it simple and to only assess those things that teams can improve upon or leadership is willing to change. It is important to get leadership buy-in for the organizational dynamics dimension and questions as teams want to see action come from their feedback. One way to do this is to review the organizational dynamics topics and behaviors with them and ask for their commitment to form an action committee to build a continuous improvement plan to address any impediments.

In Conclusion

Now you are ready to build your own assessment.  Remember to make it fit-for-purpose so it provides value to you, your team, and your organization. Take out the things you don’t need or that your organization isn’t ready to measure and add the things that are missing. Collaborate with other scrum masters and agile coaches in your network to gain great insight on new topics you want to assess.

I encourage you to take a look at the sample assessment provided in the link below. Identify topics that resonate with you and will help the teams in your organization to grow. Then identify any gaps. Are there other things you feel would be beneficial to measure? What would the maturity levels look like?

Once you have a good draft, run it by other scrum masters or agile coaches in your network. One great thing about the agile community is there are so many people out there willing to help. Feel free to reach out to me if you’d like. I’d be happy to help.

Don’t strive for perfection either, it is okay to iterate on the assessment. After your first pass at assessing teams in your organization, I recommend pulling together a working group to continuously improve your model.

Also don’t forget about the importance of consistency when assessing. Make sure results are being recorded in the same way and develop a way to merge and trend the results.

Finally, have fun and get ready to learn with your teams and build a culture of continuous improvement and growth.

About the Author

Kasie Kremenak has been working with organizations to build paths to agility for over eight years. She has worked with teams and organizations all over the world, including Budapest, Edinburgh, Netherlands, and the US. Most recently she spoke at the Agile Online Summit as part of the Women in Agile Track, will have an interview available to watch on this topic as part of the upcoming Scrum Master Summit, and will be available at the Meta-coaching clinics at the Scrum Master Summit as well. Her professional passion is in coaching, and in her private life she loves reading and gardening. She holds the following certifications: CSP-SM, CAL, ICP-ATF, ICP-ACC, and ICP-AC.