Better Metrics for Building High Performance Teams

As much as we’d all like things to be straightforward and quantifiable, there is not an agreed upon way to build and measure high performing engineering teams, let alone to track the success of software engineers. Despite the dramatic increase in software hiring recently, the industry continues to struggle to determine what exactly it is that makes a high performing team successful, let alone for an individual and their skill level within a team. Many of the metrics that already exist only really serve as a starting point for understanding performance, rather than a final destination. With modern engineering teams embracing the shift from office-based to remote work, we should also seek new ways to understand how one developer contributes to the success of the larger team. It’s perhaps never been as critical a time to consider how we onboard engineers effectively - if we want to see high performance after all - than the time we’re working in now. We must shift from the current approach that singles out vanity metrics like lines of code, bug “squash counts”, cycle time, or even amount of technical debt paid down,  in favor of a holistic approach to team productivity and software quality. Here, I’ll offer that reaching great engineering team performance is more about the onboarding and care of individual developers as they enter into a team than anything else. 

Team productivity begins with individual productivity, and all engineering teams should know the typical Time to Productivity  (TTP) for developers within their teams. Unfortunately, our industry’s tooling and frameworks haven’t set most teams up to be able to do this well. Understanding—and then optimizing—TTP is ultimately what helps developers and teams become high performing organizations. Once you understand how to effectively bring new hires up to speed as fully contributing team members, you’ll see an incredible difference in getting your product to market. This doesn’t just lead to better company performance - it results in an organization that becomes known for having a thriving, generative engineering culture that retains its best talent. 

What Doesn’t Work: Productivity Measurement Myths

There is a prevailing belief in engineering that measuring the efficacy of software engineers through cycle time and lines of code is enough to guide a team to success. I’ve found that when you center on the wrong metrics—and these are some of them—you narrow your focus in the wrong direction, leading to outcomes that demotivate and harm teams. When a company does this, they risk not only creating toxic work environments with low morale and high turnover, but also failure at large. With the average turnover rate of software engineers at roughly 15%, software engineer churn costs the industry billions annually in the US alone. 

I think it’s important to look at why these metrics are used in the first place. From a non-engineering perspective, it’s certainly an easy way to measure - there are lots of products on the market that ingest your team’s data (from places like Jira and Github, etc.) to produce visually appealing dashboards. Some even produce stack-ranked lists of engineers by “productivity” or “engagement.” Not only is software engineering more nuanced than simply creating code, these simplistic dashboards border on unethical management - certainly unintelligent management at best. Not surprisingly, VP’s of Engineering get a lot of pressure to quantify value and productivity from the engineering department for bosses or peers who don’t often have the same perspective or skillset, and these metrics are an easy way to understand activity in the field. But there’s a big issue: Lines of code rewards bloated solutions, and likewise, optimizing for faster cycle time may not solve a problem’s root cause. These metrics are often used to compare individuals against each other and teams to one another. These are dangerous paths to walk down: you will quickly fail to see the realities the team or individual may be facing (poor onboarding perhaps, or even that someone has been pulled off a project to help on something else - making the dashboard look bad.) When you begin surveilling engineers to get this kind of data, and then making management decisions based on that flawed information, you’ll drive turnover and discontent in the organization. 

What Does Work: Metrics to Watch

If we’re looking at a team with flagging velocity, it could be that their work is suffering from a lack of motivation, or that the problems they’re focusing on are simply harder than initially anticipated. Sometimes, a team is simply under-resourced or overworked. We need to look at metrics in context and focus on new ways to track and improve time to productivity. This is where it becomes critical to install and maintain a technical onboarding process that supports a new hire’s evolution to productive engineer. When poor onboarding leaves individual employees feeling behind, confused, and frustrated, the companies they work for contribute to the high cost of finding and keeping talent, and the industry overall suffers from collective burnout and discomfort.

There are a few metrics you can easily watch to help you understand your team’s productivity. In a moment, I’ll break down Time to Productivity, but to reach that fuller-story metric, you may want to regularly watch: typical pull request throughput (are there uncharacteristic delays for a team member or the whole team?), collaboration rates and quality (are team members answering questions and supporting each other?), and lead (open) time for PR’s (are there odd spikes in how long PR’s are staying open?). All of these, if taken together and within context of a specific team, can help you determine the typical time to productivity for a new member as well as the typical productivity cycles for current team members. 

Time to Productivity Starts With Onboarding

How do we move past what doesn’t work and embrace systems that do? First, we have to pull TTP apart: time to productivity is the metric for measuring the time it takes a new hire to contribute value to an organization. This is necessarily determined by how effective the onboarding process is, something often overlooked but crucial for not just individuals but those team members who will train and later rely on their performance. Teams which have ineffective, or nonexistent, onboarding programs will have high friction and lower velocity when they onboard new hires because they must spend their time supporting that new hire, instead of watching the new hire easily climb the scaffolding toward being a productive member of the team.

It is crucial to develop a baseline that lets you understand where current hires stand. Start by surveying all new hires who have started 3, 6, and 9 months ago. Ask them how quickly they think they “got up to speed”—that metric itself is different for each organization, and indeed each individual—and correlate that to the timeline their managers offer for that new hire. This is rough quantitative data on your ballpark TTP. Now, with a solid baseline for all incoming new hires, continuing to offer the same survey at 30, 60, and 90 days, and again at 6 and 9 months, will solidify the dataset. In order to optimize, we must first survey the existing situation.

If your team has just a wiki, a Trello board, or a list of links, then you don’t have an effective onboarding process. If you don’t have even that, I can guarantee your new hires are struggling (likely in silence) to find their footing and get up to speed - and they’re also dragging down the velocity of the team at large. An effective program is one in which there is a streamlined process for sharing knowledge and finding documentation that integrates new employees into both the productivity and social structures of their new workplace. Ideally, that program is supported by a team buddy and augmented with automation from deployment to evaluation. 

Challenges With Measuring TPP 

Simply creating, running, and analyzing programs and surveys are the biggest challenges here—engineering leaders have a lot to do. Often these kinds of surveys really fail to ask engineers the right kinds of questions. First, make sure you’re designing questions that get to the heart of what you’re looking to learn. Then, utilize software solutions to automate these tasks, so that you’re not constantly reinventing the wheel or remaking your baseline TTP repeatedly. Automate tasks to free yourself (and management) from the inconvenience of starting from scratch every 30 days. 

At the core, we’re simply asking questions about the engineering team’s daily work in context. Sometimes the data we’re working with will be TPP, velocity, new hire churn rates, the time it takes to fill a new role, the questions often asked by new hires, how much time a buddy spends with a new hire to help them get up to speed… there are more pieces of data than can easily fit into a simple, singular metric.

It can feel difficult to design and implement at times, but taking this data and using it to streamline improvements in TTP and velocity can completely change a technical team’s functionality and health - and often the company as a whole. Getting a handle on the dominos of hiring to onboarding, onboarding to productivity, individual to team makes this possible. 

Getting TTP Right: Interpreting Data to Improve Outcomes

With all this new data on hand, what can engineering leaders do to optimize for better TTP and better quality?  You’ll want to assess and work on improving the infrastructure that improves the experience of individual software engineers. We can focus on three specific areas for improvement; (1) developer onboarding, (2) dynamic documentation, and (3) asynchronous communication. We’re looking at problems that have historically been viewed through a technical lens rather than an organizational development lens, which in part is why tech cultures have become toxic and focus on meritocracy—I’ve seen the mindset that “smart” engineers will simply “figure it out” when thrown into the deep end at a new position, instead of creating supportive networks that allow for missteps and improve overall workplace satisfaction. When you look at productivity this way, you can see that it's not about how 'smart' an engineer is, it's about whether or not there is enough scaffolding for them to work through the learning curves of understanding new technology in a new environment.

Infrastructure Element 1: Developer Onboarding

There are four distinct “pillars” of knowledge critical to the onboarding process for software engineers: Product (what do we build), Process (how do we build it), Tooling (what’s the stack we use), and Professional Expectations (what are our engineering philosophies and norms). Despite being crucial to the success of an engineering team, these areas are easily overlooked by HR and L&D who most often find themselves in charge of the onboarding process, but lack the technicality and context for proper developer-centric onboarding. Getting a new hire up to speed is the lowest hanging fruit for optimizing TTP, but is nearly impossible without the correct learning model to support them as they acculturate.

Considering that the average tenure for a software engineer is around 18 months and that one-third of engineers seek alternative employment before onboarding completes, it’s clear that there’s a problem—if a company is not willing to fix the root causes of employee dissatisfaction - in this case, the one contributing to low team performance or slow TPP, the cycle will continue.

Software engineers should be onboarded in a way that treats their expertise as exactly that: expertise. Their onboarding process requires more work to get them to a productive, comfortable place than many other types of employees within a company.

Infrastructure Element 2: Dynamic Documentation

It’s more common than not for a company’s documentation to be out of date, and that’s often something that comes with the territory for engineering. Documentation becomes stale so quickly because engineering moves so rapidly, and it’s an arduous task for engineers to update documentation on a weekly basis. Internal docs almost always get put on the backburner in favor of approaching customer-facing deadlines, whether sprinting toward a product launch or implementing a new service.

Dynamic documentation solves this, integrating underlying code into documentation to keep up with the constant changes, instead of the other way around. This ensures that all documentation is always up to date. Thinking about onboarding and reference materials as living documents isn’t novel—these ideas have been around for decades—but they aren’t yet widely implemented enough to be considered an “industry standard”.

It’s nearly impossible to have dynamic documentation without the support of an intelligent tool, helping to automate and remind team members about it. Look for tooling that is smart enough to provide relevant answers to an engineer’s query, quickly (some companies have 1-2 day resolution times for questions developers post in Slack!) That tooling should also support rapid updates to documentation and intelligent sourcing and naming of subject matter experts. 

Infrastructure Element 3: Asynchronous Communication

As remote work continues to drive how most companies coordinate their teams, it is crucial that we develop better communication patterns. It’s easy to feel that remote team management can be done with less communication, but the inverse is true—without crafting replacements for the natural “water cooler” conversation of an IRL office, remote teams can become isolating and lack a unified company culture.