This article gives you a sneak peek into the adoption of Agile methodology at Red Hat in the platform engineering organization. Teams working on Red Hat’s flagship product, Red Hat Enterprise Linux (RHEL), a Linux operating system, are divided into functional teams called subsystems. Here we are going to focus on the Agile journey of one of the specific teams, the Identity Management subsystem, presented from the perspective of the Agile practitioner working with the team. The Agile adoption described in this article has not been uniform across the whole Engineering organization. This is by design, as it allows each team to have the time and space to develop a way of delivering work that is right for them. This also means that different teams have different levels of Agile maturity and adoption. 

At Red Hat, a unified Agile Practice team guides with the idea that we will do what is right, when it is right. This team, built of Agile practitioners, provides training, guidance and serves as Agile coaches to individual teams to help them continuously improve and choose the methodology that fits best. Some Red Hat teams use Scrum, some use Kanban, and some use a hybrid approach. 

Therefore, Agile adoption has differing levels of maturity across the product teams. For RHEL, as a whole, this journey started over six years ago. However, the Identity Management subsystem began their Agile journey four years ago and is therefore at a different level of maturity compared to other subsystem teams. Red Hat has chosen a non-religious Agile approach as we believe this is the best way to approach our co-workers and introduce change gradually. As being Agile is an ongoing process, there is no real endpoint for a team to become Agile, especially Scrum teams working in continuous retrospection and improvement cycles.

The situation at Red Hat - context for the change

At Red Hat, the goals of Agile adoption are to align our engineering teams with business objectives, improve our time to market, and increase the value of the customer experience. We began our Agile journey when we recognized that our engineering teams were challenged to connect all of their work through the product development chain back to the goals laid out by product management and the needs of our customers. What we needed was to remove the gap between the product management teams and the engineering teams. It wasn’t a problem that could be solved by simply hiring more product managers. We needed to revise the existing roles and create the right work environment. We needed to undertake an Agile transformation.

When I started my engagement with the Identity Management subsystem team, they already had some Agile experience gained through training. However, they were struggling with how to fit Agile practices into their existing environment. At the beginning of my time with the team, I started by mapping out who the players were, determined how many folks were actually contributing to the project, and identified what the dependencies were that had to be taken into account.

The core group had more than 40 contributors split between Quality Management and Engineering teams alongside a completely disconnected Documentation team. Functional People managers were trying to introduce a Scrum methodology, while also not letting go of the traditional “command and control” approach.  This led to a situation where three unstable Scrum Teams had 10+ Scrum boards, exposing conflicting priorities. Additionally, Scrum team membership changed at each iteration as people moved between these teams.

At this phase of Agile adoption, the subsystem team, overall, had reached the limits of their ability to grow on their own. They asked for help as they were self motivated to go further. At this point, their maturity level was at the learning curve phase. My initial goal was to align everyone working on the project to a shared understanding of the desired end goal.

When I first asked this group of engineers why they were doing Scrum, they answered that they wanted to improve the communication. They identified the main challenge as communication silos between “developers” and “quality engineers.” These two groups were essentially working as two separate teams with different goals and objectives.  When I came along, people managers were serving as product owners and Scrum Masters. They knew this would not scale and that the communication problems weren’t going away.  Therefore they asked for the help of the Agile Practice Team and got me.

Creating small software development teams

Small software development teams can increase the amount of completed work because of these three small software teams sweet spots:

1. Clear dependency management. The creation of smaller teams ensures that each team can focus on achieving their specific goals. A smaller field of focus increases the ability to complete work.
2. Increased team maturity. Allowing teams and people to fail fast helps accelerate the teams’ maturity level.  This freedom is easier to achieve with small teams.
3. Setting up smaller teams requires the team members to extend trust to others and empowers individual interaction. This creates a safe space where feedback is welcome and continued improvement becomes possible.

After talking to the people managers and gathering retrospective feedback from the subsystem team, it became clear that the existing organizational structure needed adjustment. We decided that we would split the existing large subsystem team into smaller durable Scrum teams. You’ll notice that I am using “we” here and not “they” or “I.”  This is because these decisions were an agreement at the team level, not a change enforced by management.

Also note that this change did not happen overnight. In the beginning, I started with  attending the various team meetings to see what dynamic they were operating. And yes, those 40ish people were having a lot of meetings. A lot of my focus was directed on who was who, and who led the majority of the conversations. I started with looking for simple patterns, e.g. are the meetings running on time? Does everyone have a chance to contribute? Is there anyone who dominates the conversation? The starting point was to get to know all of the team members as people and to understand how they work as a team.

The next step was to introduce an improvement plan to the subsystem team. This plan included reorganizing the functional teams, as separated by people managers, and the existing Scrum team structure. It is important at this point to remember that this group thought they had three Scrum teams, but, because everyone shifted teams after each iteration, they effectively had one Scrum team. Additionally, they had over 10 different Scrum boards which added to the confusion. Finally, there was no clear definition of done, which made it very hard to ever really be done.

I was fortunate to have full support of the project sponsors every step of the way. This meant that as a change agent working with this team, I had support of the larger engineering organization’s senior leadership. Their support allowed me to provide reassurances to the team that this transformation and growth effort was understood and valued, even if it had a short-term velocity cost.

The senior engineering leaders were working on their own transformation to scale Agile usage and gain increased collaboration and communication between subsystem teams. This large product level transformation was focused, in part, on connecting all of the work we do in engineering to our goals and needs. This required the ability to review and update a project-specific roadmap and a clear explanation of the overall product’s goals to the teams I was working with. 

This explanation of the larger objectives was difficult because of how we work. Red Hat is an enterprise software company with an open source development model. Therefore, the independence of our engineers is critical to their ability to be successful in the open source projects we rely on. Those projects may have goals which are not fully aligned with our product goals, and therefore we need to ensure our engineers have the ability to do right by both sets of goals. We were scaling out our Agile organization while preserving the independence of our engineers and shifting the focus to delivering value and being responsible for the product as one team. This made everyone equal.

With this going on in the larger organization, and with the collaboration of project and people managers, we formed three stable teams with dedicated product owners, and specific and well-defined technology areas. We were able to introduce missions and long term goals. This wouldn’t have been possible without the assistance and contributions of people managers. They were able to do amazing prep work during their 1:1s with their functional team members, asking for aspirations and goals. This allowed us to map everyone to the newly created small Scrum teams in a way that both made sense for the product and aligned with what excited and motivated the members.

As the Scrum teams worked, one of the interesting retrospective findings was that people were spending a lot of their time resolving infrastructure, pipeline, test automation, and similar issues. Our takeaway was to create a DevOps Scrum team within the Identity Management subsystem team to focus on CI/automated testing and automation of various manual processes. We wanted to shift this work to people who could focus on it and simultaneously create space for deeper focus for other contributors who would now be able to rely on this infrastructure instead of also thinking about it.  Additionally, this work required close cooperation with various other teams and representatives from the wider organization’s DevOps practitioners. This DevOps Scrum team provided the leadership and technical contribution that helped ultimately deliver a CI solution integrated into the larger Red Hat Enterprise Linux ecosystem and the CI systems being built by other teams. More importantly, this work also supported a shift in engineering mindset to focus on automation and automating testing.    

Measuring success

It’s easy to start with measures like positive feedback from management or timely deployment. But, perhaps the most important measurement is the team’s self-evaluation. Nine months after changes were made, we had an in-person meeting of this geographically distributed team. This gave us the opportunity to do a group exercise to collect feedback. We carefully structured it to ensure everyone participated.

We found that team members from all three Scrum teams reported increases in feelings of:

• General responsibility over the completed items
• Mutual trust 
• Sharing and collaboration

To this I would add that we had achieved a situation where everyone was comfortable expressing their opinion and participating; a huge change from where we started.

Learnings

I learned that creating smaller teams definitely supports the transition from a group of individual contributors to a self-managing team. This is critical as you cannot be successful with an Agile transformation if the team isn’t capable, comfortable and free to self-manage.

Doing Agile does not always mean being Agile. The starting state of this group demonstrated this in the way they were working. They’d had training and had wound up with a rotating cast of characters in scrum teams with 10+ boards. After I arrived, we did another training cycle on Agile. This occurred after the team had committed to doing Agile and helped everyone to acquire the tools for success. Even with their previous challenges, like many teams new to Agile they got excited. Knowledge is power. But even helping move the team members from novice to amateur still left them struggling with concepts like capacity. They, like many teams, struggled to understand what their team capacity level was. They tended to overcommit the volume of work to be completed in each Sprint. This is where a Scrum master can support by helping guide a team to maturity as they learn to deliver value and be responsible for it as a team.

It would have been impossible for me to do my work if my manager and the team didn’t trust me. I started with the trust given to me by my manager and the trust of the functional managers as a platform to build on. I layered the trust of the senior engineering leadership on top of that. Trusting your change agent, is, I would say, the starting point of scaling Agile within any organization. The next step is to make sure that the team you’re working with trusts themselves. 

Additionally, as a result of feedback collected from work with this subsystem team and others, the role of product owner in the entire engineering organization was formalized. This process empowered the product owners, who are not product managers, to make decisions.  This is done by having them work directly in the Scrum teams and with the relevant product managers.  While product owners are usually product managers, this change let us scale better than trying to change engineering and grow our product management efforts simultaneously.

Suggestions for organizations that want to go toward smaller teams

A focus on smaller teams can help any organization wanting to undertake an Agile transformation, whether at scale or in a smaller context.  The ability to mature faster, focus better, clearly define your dependencies and nurture and expand trust makes smaller teams the optimal choice.

Starting small when trying something new is always a good idea. If your company is looking to scale Agile methodology across the organization, starting with smaller teams will definitely not hurt. 

Size is often a synonym for success.  At a company level this is often true, however, with teams, size matters. As teams get larger, communication breaks down and trust is harder to establish.

Small teams scale well. They can more easily clarify dependencies and increase focus, leading to an increase in the ability to complete work.  Smaller teams can mature faster and can learn from a “fail fast” mentality more easily because they can quickly find their freedom in the group.  

A collection of smaller teams reinforces the need to trust others because your smaller team can never do everything required for a successful product shipment. You can only build your part and trust others to do the same. This trust to other teams is built on internal team trust and in turn, strengthens it. Trust is like a muscle; you have to exercise it.

In the true spirit of Scrum, you need to continually review and adapt. You cannot just form a team and leave it.  Additionally, you can’t just keep adding people to teams and expect them to continue to work. Go small to go big!

About the Author

Dominika Bula is an Agile practitioner at Red Hat, based in the Czech Republic. She is both a certified Scrum Master and a PMI certified Agile practitioner. As an enthusiastic participant of Agile and DevOps Community of Practice, Bula contributes to Opensource.com. During her journey with Agile, she has coached multiple engineering teams. Bula has a Masters degree and her education has been focused on cultural studies and psychology.  She has applied this to various support and non-engineering technology roles and has done work in business, finance, process management and service delivery.  Follow her on Twitter @domibula.

She spoke about getting more work done with smaller teams at Scrum Master Summit 2021.