The Complexity of Product Management and Product Ownership

If you, like many others are trying to understand the difference between product management and product ownership and wondering what the best practices are to implement in your team or organisation, but you are confused, then you are not alone!

I’m going to discuss how you might decide to approach your product management strategy a little differently.to avoid this confusion. We do this by operating in Complexity, applying a little Cynefin and designing your own product management. 

When hoping to introduce or improve product management in your organisation or just your team, there is a huge amount of advice and information available to you. In fact there is far too much and picking the nuanced version suitable to your organisation can be incredibly confusing. If you search questions such as ‘What is product ownership?’, ‘Do I need product managers?’ or even ‘What is product management?’ you will get thousands of answers and although there are themes, the answers are often conflicting.

Let’s consider the Product owner as described in the SCRUM guide 2020.

SCRUM guide 2020 - “The Product Owner is accountable for maximizing the value of the product resulting from the work of the Scrum Team. How this is done may vary widely across organizations, Scrum Teams, and individuals. The Product Owner is also accountable for effective Product Backlog management, which includes:

• Developing and explicitly communicating the Product Goal;
• Creating and clearly communicating Product Backlog items;
• Ordering Product Backlog items; and,
• Ensuring that the Product Backlog is transparent, visible and understood.”

Clearly this description of product ownership from the original canonical source implies that the product owner should hold the answer to many of the questions about all aspects of the product. It also ominously says this ‘may vary widely’ which is where it leaves a void that causes the disagreement and confusion we see across the internet.

With the maturing of the software industry and with an overwhelming acceptance of agility, I am still surprised at the inconsistency and overall confusion between what product managers and product owners do. - Ellen Gottesdiener

I believe SCRUM and the product owner role as a small team methodology was never meant to provide advice on how a single person (product owner) can provide coverage of the whole sphere of product management. This practice is only appropriate for the specific context of a small 7+-2 person team with complete ownership of a product.

So it seems to me that our primary source on product ownership falls a lot short of creating a consistent vision of how to make decisions to build the right thing. The following quotes are examples of the broad range of insight and advice from thought leaders and agile frameworks which is adding to the overload of information as many smart people try to fill the gap.

Roman Pilcher in his blog says:

Nearly 20 years after the publication of the first Scrum book, the product owner role is still riddled with misunderstandings. It’s not uncommon for me to meet someone who refers to her- or himself as a product owner, only to discover that the person owns a feature or the product details but not the entire product. Other times, I meet people who say they are product owners but who manage a whole product portfolio.

There is also a business aspect. Are we actually building the right thing? And it's the mindset of how to organize people that we can make decisions very quickly and learn very quickly to understand are we actually building the right things? The role of the product owner often hides the complexity of product management and upstream activities designed to ensure we build the right product - Steve Adolph

Teams become a self-fulfilling prophecy, with Product Owners replenishing a backlog and forgetting to speak to the customer. - Jon Smart, Sooner Safer Happier

The PO has a significant role in maximizing the value produced by the team and ensuring stories meet the user’s needs. This role has significant relationships and responsibilities outside the local team, including working with Product Management, Customers, Business Owners, and other stakeholders. - Scaled Agile Framework 5.0

Product Management is responsible for defining and supporting the building of desirable, feasible, viable, and sustainable products that meet customer needs over the product-market life cycle. - Scaled Agile Framework 5.0

I believe that scaling models such as SAFe have also failed to create clarity of how the concept of product management can be delivered to give great results.

Indeed when it comes to scaling product ownership Jeff Sutherland co inventor of SCRUM tweeted a study saying:

Why you can't Scale: "In summary, there is an emerging theme in the literature, namely that the original balance of scrum master, product owner and team roles are being adapted, conflated, and possibly corrupted, to suit the needs of organizations transitioning from waterfall..."

Metaphor and why are we arguing about a perfect answer?

An issue for organisational leaders considering how to design for product flow is that when someone says product ownership or product management we are immediately uncertain which of many possible definitions the person is referring to. This level of ambiguity is a constant struggle in the software world. Agile, DevOps, and Digital are all now terms which are the subject of confusion and passionate neverending debates. Product ownership/management has now joined them. Kent Beck described a similar issue in software teams when everyone has slightly different concepts in their minds when describing system components. He called this the problem of metaphor and prescribed it as a key practice in eXtreme programming. We need to take this practice of System Metaphor to our wider discussions as product delivery groups if we are going to resolve bigger issues.

To help consider some of the Metaphor surrounding the product owner function I highly recommend the blog by Roman Pilcher (an author on product management) He does a good job of creating metaphors for the key variations in product management roles.

Building our community metaphor

Our community needs to agree the Metaphor for Product ownership, Product, and Product management. But if Product owner and Product management are just nouns with fixed definitions set by frameworks then we lose their power. We need to recognise their complexity and treat them as areas of study that our community can build principles, tools and knowledge around to maximize the benefit for everyone.

We must then learn to contextualise to our specific cultures, size and environment. We do not want to debate the meaning of a practice from one methodology.

Operating in complexity - Cynefin

The challenge of product ownership, product management and value maximization is a complex problem.  The Cynefin framework tells us this is a complex problem that should be solved in iterative probe, sense, respondcycles. In this sense we need to design our own model and measure/test it and evolve it. Complexity and Cynefin tell us there is no perfect answer, no prescribed answer can be sure of success.

If you aren’t familiar with Dave Snowden’s Cynefin framework then take a read of the infoq Cynefin mini-bookor watch Kaimar Karu otherwise a quick search will provide you with many wonderful summaries.

The key understanding here is that deciding how to manage products within a team or an organisation with many agents (stakeholders, business owners, customers, designers and builders) is a Complexity problem. In Complexity several key rules apply:

• There is no ‘right’ answer and no ‘best practice’
• The success of any process we design will be affected by the people using it
• The process will have unintended consequences
• What has worked for others will not work for us except by accident
• Guaranteeing a future state is impossible
• To move towards our goal we must experiment and iterate

This means we must not copy product management or product ownership patterns from other organisations or frameworks. We must learn to operate in complexity and iterate. Many of you will spot the similarity with PDCA or Build, Measure, Learn from Lean startup:

1. Identify the outcomes we want
2. Diagnose where we are in comparison to the outcomes
3. Research and design an experiment
4. Trial a pattern in our context
5. Back to step 1 - Adjust it again when it goes wrong / doesn’t achieve the full goals (and it will go wrong)

Your contextualised product management - Identify the outcomes you want

If you are wanting to change the way you manage your products and services then it makes sense there must be some element of your team or organisation which you are hoping to improve so that you get better results in your effectiveness or efficiency. So the goal here is to name the outcome improvement!

Here are some examples of the kind of outcome and measure you need to pick from.

Note: You can’t get them all at once, as Complexity tells us that any change you make towards one outcome will affect the others in an unknowable way!

Your contextualised product management - Diagnose where we are now!

The next step is to determine what are the weak and strong elements of our product management value stream and see which elements are possible places to improve. Following is a list of possible elements you will have to consider when looking at the current state within your team or organisation. This list is definitely not complete so use your own team to brainstorm other possible areas for assessment.

I recommend gathering a diverse team to do some data gathering and sensemaking. Then categorise each of these items and others you identify as red, amber, green. To determine if something is green will require making sure you have specialist expertise on the team to help shed light on possible levels of maturity or unknown unknowns someone close to the action may miss.

Product & Service Orientation  What stance does our leadership take towards delivering value (functional, product, service, customer)?

Innovation Orientation. How do we allocate investment in innovation into our delivery priorities?

Product market fit. How would you assess your organisation's product market fit? How do you position, brand, and sell your products and services externally?

Portfolio leadership How would you assess your leadership's portfolio management and prioritisation?

Product & Service Team Structure What striping (function, product, service, customer) does our product & service team structure map to?

Prioritisation How are priorities set across the organisation and within teams?

Measurement & Incentives Have you got a system that is assessing whether we've created the right product by clever metrics like outcomes, and customer feedback  (versus vanity metrics). How do you incentivise people to understand the customer?

Customer-Centric Learning Cycle How much are we incorporating user feedback into the product and service design and delivery process?

Managing Work Distribution Is there a structured process to request new work/features, have it scheduled and managed, and to be completed?

Innovation How well do we integrate product and service innovation into our organisation? Are there teams dedicated to innovation, is everyone involved, is it through bureaucratic PMO processes?

Roles and People What Roles, Skill level and People do we have to achieve these outcomes?

Your contextualised product management - Prioritising an improvement initiative

Once you have assessed the diagnostic areas above you should have good enough data to choose a selection of improvement initiatives noting that you cannot change everything in one go (see complexity and Cynefin). Be careful to pick a change which is not too ambitious (next adjacent) and possible to test in a sprint window of about 3 months.

We should commit to designing process changes in small experimental iterations, checking our measures are improving each stage. Consider possible initiatives like these to focus on first depending on where the weaknesses showed in your diagnostic:

Initiatives focussed on Product Outcomes

1. Align product choices more clearly to the organisational commitments, vision and strategy
2. Clarify the process by which innovative ideas become commercial products. Communicate this to the whole organisation.
3. Clarify a WSJF (Weighted Shortest Job First) formula for prioritising choices between different products, features and competing opportunities. Communicate the different variables that make up the weighting and use this to create alignment.

4. Design a process to measure and check product-market fit and delight our customers

Initiatives focussed on Skills and team makeup

5. Hire a user researcher, and a service designer, assign them to a team and together with the developers collaborate with the customer to understand the real needs to be met.
6. Run a delegation board exercise and assign levels of product decisions between team, manager and key product leaders. Determine who says ‘No!’ when it’s a question of quality or sales deadlines. Communicate the decisions.

Initiatives focussed on Process

7. Pick a key value stream in your organisation, map the value stream and identify the key constraint. Then work to remove that constraint, so that you can bring changes to market faster and at higher quality.

Any change initiative you design will need to include change strategies for the leadership, systems and practices of the organisation or team. Think about these factors.

Leadership

We need leadership to provide a clear vision, principles and commitment to how the organisation should prioritise orient, innovate and design products.

Systems

We need to create aligned and coherent systems such as prioritisation, governance, risk, finance and incentives that support teams to follow the vision and principles set by leadership.

Practices

We need to employ practices that deliver measurable results aligned to the overall commitments made to customers and follow the principles laid down by the leadership.

In Conclusion

• Avoid debating terminology - Whether you call a role or a function product owner is not relevant, focus on the outcomes you want and create your own evolution for your specific context.
• Avoid fixed methodologies and frameworks - Fixed methods and frameworks cannot work in complex environments and if they worked for someone else you cannot know the combination of complex interactions that made it work. As a rule, do not copy or be sold by patterns not generated in your organisation.
• Investigate and read widely to understand all elements of great product delivery - To have the best chance of creating a successful evolutionary change, you must understand the key principles at work from psychology to flow efficiency.
• Examine the strategy, systems, processes and practices you are using to drive products and services - See the whole, make sure you apply some systems thinking and theory of constraints so that you do not miss the key constraint to work on.
• Design an evolutionary improvement initiative made up of iterative experiments - For Complex situations Probe, Sense, Respond. Build, Measure, Learn. Plan, Do, Check, Act.
• Implement interventions and measure, then accelerate, dampen and pivot as necessary

Recommended reading list

• How to lead in product management
• Discover to deliver
• Inspired: How to Create Tech Products Customers Love
• The Lean Startup: How Today's Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses
• Badass: Making Users Awesome
• This is service design doing
• Managing The Design Factory: A Product Developer's Toolkit
• Creativity, Inc.: Overcoming the Unseen Forces That Stand in the Way of True Inspiration

Other definitions if you are curious

Product ownership from extreme programming - Ron Jeffries

In Extreme Programming, every contributor to the project is an integral part of the “Whole Team”. The team forms around a business representative called “the Customer”, who sits with the team and works with them daily.

• Core Practices: Whole Team

Extreme Programming teams use a simple form of planning and tracking to decide what should be done next and to predict when the project will be done. Focused on business value, the team produces the software in a series of small fully-integrated releases that pass all the tests the Customer has defined.

• Core Practices: Planning Game, Small Releases, Customer Tests

Extreme Programming teams develop a common vision of how the program works, which we call the “metaphor”. At its best, the metaphor is a simple evocative description of how the program works, such as “this program works like a hive of bees, going out for pollen and bringing it back to the hive” as a description for an agent-based information retrieval system.

Sometimes a sufficiently poetic metaphor does not arise. In any case, with or without vivid imagery, XP teams use a common system of names to be sure that everyone understands how the system works and where to look to find the functionality you’re looking for, or to find the right place to put the functionality you’re about to add.

About the Author

Douglas Talbot is an experienced technology and product leader, specialising in creating and leading teams building complex, innovative products across tech, engineering and science boundaries. He has scaled agile and digital approaches to over one thousand people, distributed internationally, with 24/7 operations. He is known for a building engineering teams, great cultures, and speaking on leadership, organisational dynamics and design.