At the fifth ‘Agile on the Beach’ conference, held in Cornwall, UK, InfoQ sat down with Melissa Perri, founder of ProdUX Labs. Perri presented a talk entitled ‘Continuous Product Improvement’ at the conference, which questioned that with the widely implemented technical practices of continuous integration and continuous deployment, why is it that continuous product development is not yet mainstream?
InfoQ: Hi Melissa, thanks for taking the time to join us today. Could you tell the readers a little about yourself, and also briefly introduce what you were presenting about at 'Agile on the Beach'
Perri: Thanks, it’s great to be talking to InfoQ about something I’m really passionate about! I’m a Product Management and UX consultant and coach. On the coaching side, I teach Product Managers the skills they need to create great products. On the consulting side, I help teams answer the question “should we build this?” instead of “can we build this?”
We spend so much time talking about how we are going to build software. At conferences, in meetings. Everywhere. We talk about sprint planning, our code bases, and shipping features as fast as our fingers can fly to submit that pull request. We get so lost in “How” that we forget to ask “Why?” Why are we building this? This is the most important question you can ask when building a new feature or product. I make sure my clients and students don’t forget the “Why”.
At Agile on the Beach, I talked about Continuous Product Improvement. It’s my way of helping teams improve their products scientifically. The key to Continuous Product improvement is the Product Kata, which is based on the Toyota Kata method of Continuous Improvement. This practice makes the entire company’s job to improve the way they work. Everyone, from the CEO to the janitor, figures out how they can further the company and reach a common goal. This is what made Toyota so successful. I’ve taken these concepts and adapted them for Product Management.
InfoQ: Your talk today discussed 'continuous product improvement', and you questioned that given the ubiquity of continuous integration and continuous delivery, why is this not also mainstream? What do you think the answer to this question is?
Perri: I think the biggest difference is that we spend a lot of time figuring out how to make development better with processes like Continuous Integration and Continuous Delivery, but as I said before, we neglect the product puzzle pieces. I think that’s because the Product Manager or Product Owner role is very misunderstood.
Many companies see this person as the one who determines requirements, assigns user stories, and makes sure the developers are building to spec. Maybe, they run an A/B test or two. That’s if you’re really advanced. These functions are definitely an important piece of the role, but they miss the big picture.
A Product Manager is a facilitator. As a facilitator, you don’t come up with all the ideas yourself. Instead, you encourage the team to solve the problems and help steer them in the right direction. The Product Manager is responsible for finding the best product opportunities for their business.
This is what Continuous Product Improvement is all about. It allows the team to determine how to optimize features, start new product lines, and reach company goals by putting the reigns in their hands. When you work this way, your team feels more ownership around the product. They get excited to build it.
We’re always talking about how to improve our development efforts because we’ve associated success with feature releases. It’s tangible, something that we can see progress on directly. This makes it comfortable. We need to all take a step back and realize it doesn’t matter how many features we build if no one is going to use them.
InfoQ: You also talked about the 'Product Kata' in your talk. Could you explain the implementation and benefits of this process please?
Perri: Sure. The Product Kata is the process that systematically steps you through improving your products and aligns your team around a common goal.
First, you need to get some support from management. Whoever is setting the vision for the company or the product sets a goal for your team to work towards. For example, “We’re going after the enterprise market for our clients now. In the second half the year, we want acquire at least 10 enterprise customers.”
These goals are usually lofty. They’re ambitious and we’re not 100% sure how we’re going to get there. We need to break them down into achievable, smaller goals called “Target Conditions”. An example of the first target condition for our enterprise goal could be “Sign up 1 enterprise customer as a beta tester.” This is something we achieve faster and it’s easier to wrap our heads around getting there.
When the team understands the direction and where they are heading, they then start the Product Kata. The Product Kata is a series of questions that we ask ourselves every time we take a step towards achieving our Target Condition.
1. What is the current state?
2. What is the obstacle we are currently tackling?
3. What experiment can we run?
4. How do we measure success?
After we walk through those steps and run the experiment, we evaluate our progress and ask “What did we learn?” We use this information to run the next experiment. If we’ve reached the Target Condition, we set the next one until we achieve our overall goal. If not, we continue experimenting and learning until we do.
For a more in depth example and explanation, check out my blog post on it here: http://melissaperri.com/2015/07/22/the-product-kata/
InfoQ: Designing experiments and establishing metrics of success appear vital when using the continuous product improvement approach. Could you offer any advice on these topics?
Perri: Metrics are extremely important to running the Product Kata, and to making good products in general. I typically see companies who are not measuring success at all, or who are measuring it the wrong way. A great example is when companies say that their goal is “To release 2 new features next year.” Anyone can release two new features. That doesn’t mean they’re good features.
It’s important to base metrics on outcomes, not outputs. Review all your metrics and ask, “What am I achieving for the business or customers here?” If it’s not making an actionable difference for either, then you should change that success metric. These metrics can be quantitative or qualitative, and you should have a nice mixture of both.
Another key point of a good metric is that it is actionable. An example of a bad metric would be “Number of app downloads.” You can’t do much with this. It will always increase until no one wants your app. If you say your metric is “To increase the conversion from views to downloads month over month by X%”, you can start experimenting around what you can do to achieve this.
InfoQ: What are the biggest challenges you see with implementing continuous product improvement?
Perri: Teams struggle with creating good experiments and measuring them at the beginning. It takes practice to learn how to make great experiments. You need to be able to analyze and reflect from past experiences to see what you can do better. So, the Product Kata has to be done repeatedly, just like standups.
Also, the team has to be invested or they’ll try to sabotage it. One team I coached made their first experiment “We get a piece of candy when we update something.” No joke. But, they weren’t sold on the process. It was something that was mandated to them. Eventually they came around and enjoyed this way of working but it took a long time. If you want a team to create great experiments, you need to make sure they are willing to try. That’s the biggest challenge.
InfoQ: Many thanks for your time today. Is there anything else you would like to share with the InfoQ readers?
Perri: I’ve seen this as a great way to improve existing products and create new product lines when there’s a foundational business. But, this isn’t the route to go if you are creating a startup from scratch. You have to analyze your unique situation before implementing any new process. That’s the only way to pick the best ones. With that said, I hope you’ll give this a try and if you need any help or have any stories to share, please reach out to me!
The video recording of Perri's "Continuous Product Improvement" Agile on the Beach talk can be found on the conference YouTube channel.