User Stories Are Placeholders for Requirements

Agile Teams are responsible for Business Analysis and it is the job of members to Elaborate User Stories into Requirements

It doesn't matter what methodology you follow (or pretend to follow) Agile Scrum, XP, UP or Waterfall, poor requirements will always hurt a project more than any other single failing. You only have to check the literature to see this has been true throughout the brief history of software development.

User stories are a convenient way to prioritise and reference those requirements but don’t let their dual purpose mislead you – you need to know when to elaborate and how far to drill-down. Too much and you won’t have enough time left to deliver, too little and you won’t be able to produce a viable delivery plan.

It can be difficult to change from a Waterfall approach where ‘business analysts write big requirements up front’ to the Agile practice in which requirements are prepared ‘just in time’, and are the responsibility of the entire team. The secret to success in Agile is ruthless management of scope.

Planning Meetings are all about Scope

Scrum’s Sprint Planning meeting (and its various project equivalents) is a critical time for requirements. This meeting is really an interface boundary where business people negotiate the hand-over of their high-level requirements to the problem-solving team. It marks the intersection between the problem domain and the solution domain. 

By detailing what’s in scope and what’s not, immediately before starting the next sprint of work, planning forms the delivery contract between the development team and the Product Owner. By the end of the planning meeting there can be no hidden assumptions or lurking misunderstandings about the requirement, the solution, or the criteria for acceptance.

I’ve seen many managers of Agile teams invest time to attend their team’s planning meetings, and they are right to do that, but what should they bring to the table at that meeting? What are the tell-tale signs of effective planning and are there any tell-tale signs that the ensuing sprint may become compromised? 

Like most things that appear easy, it takes an enormous amount of work beforehand to run an efficient planning meeting. It is the job of every team member to prepare for that meeting so that the Product Owner is able to make informed ‘in or out’ decisions. It’s the manager’s job to support the team in their preparation for that meeting.

On a recent coaching assignment I was fortunate to work with several different Scrum teams, their managers and executive sponsors. All knew the recipe for Scrum and all were willing practitioners. Performance varied immensely between the teams, but several factors became rapidly evident.

• Manager’s behaviour during the Planning (and Review) meetings had a direct impact on the performance of the team. When managers knew about the topic under discussion they were able to add value to the discussion and the team was able to make better progress because of their input.
• When managers and team members did not agree the mechanics, roles and responsibilities of the process as applied to their unique combination of people, processes and constraints, the results were disappointing to all.
• The most successful teams had managers, Product Owners and Scrum Masters who were focussed on the overall success of the team.

Gaining agreement on exactly what is and is not included in each story was demonstrated to one team when a senior manager received the news that a deployment story had been completed, and casually asked if that deployment included all six modes of transaction. Learning that only two modes were deployed, because that was all that had been explicitly referenced during planning, turned a potential reason for celebration into a perceived failure. This experience led the team to becoming very specific with the scope of their stories.

The Business Analysis Role

In the Guide to the Business Analysis Body of Knowledge® (BABOK®Guide) the International Institute of Business Analysis (IIBA) states “The business analyst is responsible for eliciting the actual needs of stakeholders… facilitating communication between organizational units… aligning the needs of business units with the capabilities delivered by information technology, and may serve as a "translator" between those groups.”

Whilst everyone understands the importance of requirements, no named role in an Agile team has the job responsibility for writing them and in practice, developers, testers, analysts, subject matter experts and project managers must use whatever methods they can to do the work of the business analyst.

When should you do Requirements Analysis in Agile?

Team members often ask: ‘when does requirements analysis take place in Scrum?’ The answer to this question is neither satisfying nor helpful, as it is “from the time you write the user story title, right up until you declare the story as ‘done’ (according to the team’s current definition of done)”.

Compared with coding, which is precisely bounded by the start of the sprint to the end of ‘done’, requirements analysis seems to be an impossibly elastic activity, emerging from the analyst’s existing understanding of the business and continuing to the end of the life of the application. Gaining agreement on acceptance criteria is therefore a significant marker in the requirements journey of each user story.

The vagueness is because requirements emerge over time and from engagement with people. The IIBA uses the terms ‘elicitation’ and ‘requirements analysis’ to describe the work of understanding the actual needs of the stakeholders and progressively elaborating, defining and validating solutions. Clearly, both require face-to-face time spent with stakeholders: users, sponsors, product owner, and various other members of the development team in order to reach a shared understanding of the problem and the solution. Being Agile, this work is done ‘just-in-time’ and as prioritised by the Product Owner, who must understand the relative business value between items of the backlog.

Because Agile requirements are emergent, a better question to ask is ‘what business analysis work needs to be done by the development team during this next sprint?’ This is powerful because it ensures analysis effort is invested only on the current high priority stories, while stories of lower business value, and therefore lower priority, remain in the backlog.

All features are important, but since the team is only able to deliver one feature at a time, the business is required to make a choice between features A and B, or process flow X, Y and Z. Clearly, some prior analysis must have taken place in order to have identified A, B, X, Y and Z, hence the emergent nature of iterative development.

A worked example should help demonstrate the way this works in practice.

Lifecycle of a User Story

Most stories will start life as something quite epic, such as ‘Become Agile’, but such an ‘epic’ is far too chunky to be achieved in a single sprint and so it has to be split or ‘decomposed’ into smaller stories, which might include: Write User Stories; Deliver Monthly; Fail Fast; and so-on. These smaller stories (they are merely titles at this stage) are inherently Agile because they can be organised into an actionable, prioritised backlog of work. That’s important.

It’s perfectly acceptable to leave them as titles for as long as possible, but since the stories at the top of the backlog are the subject for the next planning meeting, it is better to add detail as their position rises. Let’s take one and write it out in the format:

     As a <type of user>, I want <some goal> so that <some reason>.

A story is a simple statement of functionality that is usually written like this:

     As someone involved in software development, I want to write user stories,
     so that everyone has a clear understanding of what is expected to be done

This format was originally developed by UK coach Rachel Davies and then popularised for use in Scrum by Mike Cohn, who wrote a book on the subject. The format describes:

• The [primary actor, user role, or ‘persona’]
• A single [action or function that the actor wants to happen]
• That actor’s main reason for doing it, or expected benefit

Most teams follow the first part, but I’ve noticed some are reluctant to add the reason in the ‘so that’ part. They say there’s no point, but this is the link with the business need; it provides the answer to the question ‘why are we paying you to develop this feature?’

According to Bill Wake, we should INVEST in stories. Every story should be: Independent; Negotiable; Valuable; Estimable; Specific and Testable. INVEST has proved to be an excellent checklist for stories, and is broadly compatible with the IEEE’s recommended practice for (traditional) requirements specifications which are: a) Correct; b) Unambiguous; c) Complete; d) Consistent; e) Ranked for importance and/or stability; f) Verifiable; g) Modifiable; h) Traceable.

The story above is not yet a ‘good enough’ requirement according to INVEST, as it’s neither specific nor testable. Plus it is ambiguous – open to interpretation. Who, exactly, is "everyone"? Does it include the family living in the apartment below, or just the development team? What about a project manager who is outside the development team, but is tracking progress for budget purposes?

Besides, how can we measure "understanding"?

And what, exactly, is the definition of "expected to be done"?

So we must extend the story by writing story-specific acceptance criteria that will help answer the most obvious questions, even though we don’t have all the answers and will almost certainly need to consult others.

User Story Elaboration with Acceptance Criteria

By writing acceptance criteria, the team is able to recognise the various pathways that exist within the story, in much the same way that a traditional flow chart that documents a use case will reveal several different paths. The first story, which is also likely to be the highest priority, describes the successful, sometimes known as the ‘happy’, path.

The format for acceptance criteria is, ‘This Will Be Done When’ which you may wish to abbreviate to TWBDW. For the above story:

As someone involved in software development, I want to write User Stories, so that everyone has a clear understanding of what is expected to be done

This will be done when / TWBDW:

     1. A user story is available to anyone within our organisation that wants to see it
     2. The development team have understood it well-enough that they are able to estimate the work required to develop a solution for it
     3. At least 10% of our requirements are written as user stories
     4. ??% of all teams are using user stories (this may not be part of the requirement, so we will ask)
     5. Only people within our team can add or change a story

You can see that the first couple of criteria are quite obvious. They came to me (the writer of this story) very easily but then I thought of the quantity and made a guess that 10% would be ‘good enough’. Which led me to wonder if we should seek to apply it to any other teams, but that’s not something our team can decide. Finally, I realised we should consider editing rights.

So far this is the work of just one person and is typically one-dimensional.

If stories are presented to a full team at this early stage, you will see that meetings quickly become tedious and ineffective. Basic questions will be raised, many of which cannot be answered by anyone present. Trying to prepare a story with the full team looking-on is wasteful and frustrating plus it’s impractical to find the people that need to be consulted, and if you find yourself in this situation it would be best to acknowledge the lack of preparation, and move-on to the next (prepared) story.

Collaboration is essential for Agile teams

Prior to any team-planning meetings the next step is to have a conversation with another stakeholder, perhaps the Product Owner, or a subject matter expert, or another member of the team. This is known as ‘backlog refinement’ in Scrum and ‘elaboration’ in business analysis. This collaboration allows people to analyse and elaborate the requirement, consider other dimensions, criteria and scope. Two people working together are better able to concentrate and produce higher quality output than one – it is the business analysis equivalent of pair-programming and we now know that collaboration is an enabler of Agile delivery.

After just a few minutes with a colleague, I had two new criteria:

     6. Everyone on the team should be trained in user stories
     7. The training must take no more than two hours
     8. It should take no longer than 20 minutes to write a user story

Great. See how that other mind works differently to mine and together we are more powerful?

One of the teams that adopted this technique preferred to pair a developer with an analyst to help support the Scrum concept of no job titles other than team member. Another team went further and tried pairing people from different parts of the business, including a remote colleague, whenever possible. Although they had always worked individually prior to this change, both teams enjoyed the experience. One unexpected benefit of pairing-up was that more people became involved in the elaboration process and they felt more personal ownership of the stories they prepared. I noticed that when the time came for the story to be presented to the whole team at the planning meeting, the effect of a change of presenter brought new energy to the meeting, and took pressure away from Scrum Masters who had previously presented every story.

Sprint Planning Meeting with a Properly Prepared Story

We’ve seen that two brains are better than one when it comes to requirements elaboration and writing acceptance criteria, but what happens when the story is presented to the whole team at the planning meeting?

Imagine the scene: it is the sprint planning meeting, and our story is about to be analysed and validated by the team that is being asked to design and deliver the solution for it. They must assimilate the requirement, negotiate and estimate the work involved and accept or reject the story for the forthcoming sprint.

Sprint planning is Scrum’s assurance mechanism. It chases-out hidden assumptions and validates the communication between the business and the development team. It’s a process that ensures sponsors provide adequate face-time to their developers.

We know that any team member can write and present a story to the team and it is the conversation that follows that is important. Ours goes something like this:

     “We don’t need a new way, we are very happy with our Use Case documents” says one member of the team on hearing the user story ‘story’.

     “I don’t have the time” says another, looking stressed.

     “We used Agile on a project at the place where I used to work and actually found it pretty quick easy to do” offers another.

     “How big should a story be?”

     “How do we break stories down?”

     “Will we still have to write Use Cases?”

     “What happens to the stories afterwards, and will they be part of our documentation?”

We’ve all encountered this sort of cross-questioning, but here it’s exactly what we want to happen. As the team rips-into the requirement, they are testing their own understanding of its validity; they are trying to make it fit within their knowledge and experience; and examining the problem by bouncing-off each other’s ideas and feelings.

They are having a proper discussion in which the team is actually examining the user’s requirement. Once they’ve done this, they will understand what the user wants, and why, and why the user doesn’t want something else. (OK, my example breaks-down here, because the team are the subject of the story. That just goes to show you the problem of writing a story with a team member as the primary actor!)

The team have decided that points 4, 6 and 7 are not part of the core story and rather than lose them, choose to split them out to form new stories. This keeps stories Agile so they can be raised or lowered in priority in finer granularity. In this case, the training stories (6 and 7) should happen first, followed by the main story. Spreading to other teams is a nice idea, but may take a long time, or may never happen. It does no harm to keep it as a story though, but it will stay near the bottom of the backlog with the other low-priority stories.

Stepping-back and looking at a wall that is now a regular patchwork of coloured sticky notes, the team’s ‘Mr Use Case’ remarks that they have managed to expand a ‘brief use case’ into several prioritised and scoped requirements, something that he would expect to spend at least day or two doing by himself, before presenting his results back for the project manager to update and move to the next stage in the process. This new process seems to be an improvement.

As we leave them to the rest of their meeting it is apparent that everyone present will be able to start working on this story – they all know what it is about, have all discussed it and have understood the scope and success measure of each aspect of the deliverable.

How to “Read” the Planning Meeting

If you really want to know how a team is doing, just observe their Sprint Planning meeting and compare what you see to your knowledge of how the process should work. You don’t need to be an expert in Agile for this, just interested in people, and aware that this meeting is driven by a requirement that might be called a story, use case or product backlog increment.

Based on my observations of teams, I developed some check-points to help managers “read” this meeting. Some of these may be applicable to your teams.

Concerning story elaboration:

     - Has the story been split-down into an independent and valuable story?
     - How well does the team analyse the underlying requirement?
     - Have they identified any external dependencies, and if so, do they plan to handle them as separate stories or are they likely to cause blockages?
     - Does the team’s analysis reveal additional stories and acceptance criteria and does that help define the scope of this story?
     - As the planning for the story progresses, is the proposed solution “real” for everybody (including you)?
     - When they break the solution down into tasks, do those tasks seem to pop-out obviously and naturally?

On estimating the story:

     - Is everyone engaged in reaching a reasonable estimate collaboratively? 
     - If they place an actual time estimate on the tasks, do they add-up to a reasonable proportion of the sprint, given the value estimate of the story?
     - Is everyone clear about the unit used for estimation?

About the team interaction:

     - Note how junior members are treated by the others. Is it important that everyone understands the requirement well-enough to participate fully?
     - How well has the team prepared? Is their rate of progress consistent with their initial expectations?

Without first-hand and team-level experience of Scrum, managers observing planning may feel divided between helping the team to reach the right decisions to deliver the results the business expects and supporting their team’s adoption of Agile methods. In many cases, it may seem easier to change the rules (the game) of Scrum to accommodate existing governance, structure and behaviours, but by changing those rules, one loses the right to expect the delivery benefits of Agile.

Conclusion

Agile delivery is a matter of prioritisation which is managed by stories. User stories are placeholders for requirements which emerge and are expanded as needed.

Successful Product Owners and their teams produce a steady stream of deliverables by limiting scope to the two or three highest priority features that the team is able to confidently analyse and predict. Items removed from scope remain as stories and await their turn in the backlog.

Every team member has a part to play in gathering and validating requirements so that they emerge from the planning meeting (or equivalent) with a shared understanding of the requirement, enabling each to perform valuable work from day 1 that will help the team reach its agreed goal.

Managers and leaders prepared to use Agile values and principles (knowingly or not) to support their teams can expect to be rewarded with Agile’s performance improvements including predictable and frequent delivery.

About the Author

Russ Lewis, is an Agile expert and coach. He leads Agile adoption at executive, manager and team levels, helping enterprises improve “Organisational Agility”.
Russ architected and led Transport for London’s Contactless Fares payments system and the expansion of Oyster card to National Rail; supported the Agile transformation at Amadeus SAS; speaks at conferences and writes a blog.