Meet my friends Jane and John.
John is a long time analyst in a large company, and is responsible for requirements elicitation in new and existing software products. John uses SRS (Software Requirements Specifications) as a work tool to record all the client’s requirements for the specific product being developed or maintained.
Jane is a developer in the same company. She usually receives from John the SRS to start making technical analysis and designing what is to be implemented. When this analysis is completed, she starts coding.
Both my friends John and Jane hand out requirements and design documentation for approval; in John’s case, before delivering specifications to Jane, and, in Jane’s case, before starting to code.
Recently, John and Jane’s company adopted agile methodologies in project management and software development, and the way things were made changed. No big up front requirements and design were to be made, specifications and developments were to be split in small chunks of information, and big word documents like the ones used for several years were to be discarded. The way developers coded also changed, and things like designing tests before implementing, and naming functions with longer names were being encouraged.
As expected, John and Jane raised a whole lot of questions:
- How can we start developing if we don’t know in advance what to develop?
- Where do we record functional data, if not in the usual SRS documents?
- How do we know all the details of what is to be developed?
- Where do we document specifications and code for future maintenance?
- When do we document, if there is no documentation phase?
The adoption of agile methodologies in project management and software development has experienced a rapid growth in the last decade and is expected to keep growing. In transitioning to the agile way of working, many Johns and Janes throughout the world pose the same questions on what appears to be such a loose approach to development and is definitively a different, less traditional way of doing things.
In the middle of all the differences in the way companies begin to work when transitioning to the agile mindset are issues relating to documenting.
This article focuses on why, when, how and where to produce technical and functional documentation.
Documenting and the agile mindset
The Agile Manifesto states that we value:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
Although it does refer to documentation, agile principles do not give any rigid guideline on how to document.
Therefore, what is expected to be produced in terms of documentation in an agile managed project?
The principles behind the agile mindset focus on delivering value to the customer. That means that the time we spend developing the product should be spent doing something that adds value to the customer, avoiding wasting time in tasks that add little to no value. This also applies to documentation.
In the typical waterfall approach, documentation is a predefined phase that consumes a lot of time. Transitioning to an agile approach means that we have to rethink the way we document, in order to avoid wasting time producing a deliverable with questionable value.
Does this mean we should not document anything? Not at all.
Another of agile principles lies in adapting to change. That means that we do not plan too much in advance because things do change along the project. So, the main focus is always in just-in-time planning. This applies also to documentation. In order to avoid wasting time, documentation in agile should occur only when it is necessary.
But, how do we know when it is necessary?
Do you really need to document it? – WHY to document
When coming from the Waterfall world, the natural tendency is to bring along habits of defining big requirements up front, which can lead to:
- documenting unnecessary details
- documenting traditional specifications that are agile just in name
In order to avoid that, there are some guidelines to help us decide if we really need to document it.
To avoid wasting time in documenting, ask everyone involved these questions:
- What is the documentation needed for? To support development? To provide functional references?
- Who is the target audience of the documentation? What person? What department? Is it the client? Or future maintenance teams?
- How will the target audience use that documentation? As a reference? As a manual?
- How much it will cost to produce this documentation? How much time will it take to produce it? By whom?
The key to deciding whether we should document or not is defining just-enough documentation. Find the balance between what stakeholders really need and produce exactly that. No more, no less. Just enough.
What, where and when to document
After agreeing on why and how much to document, define what documentation will be produced BEFORE you start producing it.
What
Align with the target audience of the documentation what will be produced. Tutorials? Training documentation? Maintenance support documentation? Business rules documentation? System description documentation?
In order to decide what to document, define what technical and what functional information will be produced, in each phase of the project:
- What documentation will be necessary before the project starts?
- What documentation will be necessary during the project?
- What documentation will be necessary after the product has been implemented and rolled out?
Where
It’s useful to keep available documentation in an accessible and yet constantly updatable repository. Using word documents or similar formats can cause documentation to be stale or out of date.
Documentation should be available as the project rolls along and in a permanently updated support, available to the target audience of that documentation.
When
Unlike what is expected to happen in waterfall approaches, in agile approaches there is no documentation phase.
Therefore, as features are being added incrementally, the documentation should be incrementally produced as well, evolving along with the development of the product.
How to document
At this point we should have decided:
- The just-enough documentation to produce
- What to document
- Where will the documentation be stored and how it will be available
- When it will be produced
Now, let’s explore some suggestions and best practices on how to document in each phase of the project.
Project Kick-Off Documentation
In the beginning of the project, little documentation will be necessary, since the real work is about to begin.
When it comes to technical documentation, define two or three high-level architecture diagrams of your choice, to identify high-level components of the solution to implement.
In the functional field, main Epics identifying the main characteristics of the product to develop should be enough.
Since the project will be managed using agile approaches, no upfront design will be demanded, so in this phase defining just-enough information should be enough to get the team started.
Project Documentation
During the project, while the product is incrementally being developed, two kinds of documentation may be produced:
A. Just-enough requirements – as an input to the Product Backlog
B. System and business rules implemented – as an output of each sprint
Each kind of documentation as its own target audience:
A Target Audience: The team
B Target Audience: Stakeholders and eventually maintenance teams
The documentation provided should respond to the needs of both target types of audience.
Functional documentation
The most common support for functional documentation in agile managed projects is the User Story. In order to give developers the requirements on how to implement, User Stories aggregate just-enough information, described in the following format:
- Description: In the form of “As <actor> I want to <action> In order to <added value to the customer>
- Acceptance Criteria: The criteria accorded between the team and the Product Owner to define if the story is completed
User Stories must be accepted by stakeholders before being planned in the sprint, which means that at the point they are ready to be planned on the sprint, they are supposed to be a reliable source of information (requirements) for developers to know what to implement. This should provide just-enough documentation for target audience A.
As the User Story is being implemented, the team might decide on new acceptance criteria, which is then added to the User Story.
Remember that in agile, User Stories are not supposed to be exhaustive requirements specifications on a small chunk of information, but instead are expected to provide the guidelines on what to implement, as a promise of future collaboration between the Product Owner and the team on what to implement.
When the User Story is implemented and accepted by stakeholders, then it is considered to be accepted. At this point, it should move from the Product Backlog - the set of items describing features to implement in the Sprints - into the Product Increment - the set of Product Backlog items completed during a Sprint and all previous Sprints.
As User Stories are being completed incrementally, its implementation provides the documentation necessary for target audience B, as shown in Figure 1.
Technical documentation
During development, developers should use accepted development best-practices, as part of Agile Development Methodologies, like TDD (Test-Driven Development) and BDD (Behavior-Driven Development), as well as well-structured code, readable by non-technical folks. This should provide the technical documentation that will describe the code.
Up-to-date functional and technical documentation
Since the code is the most reliable source of up-to-date business rules and system implementations, the best way to have useful documentation is to base it on the code itself.
A way to do this is by implementing Living Documentation.
A living document or dynamic document is a document that is continually edited and updated. The concept of Living documentation relies on integrating documentation in the code based on the acceptance criteria written in a Domain Specific Language, like Gherkin used in Cucumber. By using tools to facilitate the continuous build of living documentation, acceptance criteria written in User Stories are implemented as part of the code, and outputted as a readable format at an updated and typically online location.
Some tools that provide mechanisms of living documentation are Cucumber, Concordion, FitNesse, SpecFlow, Pickles, amongst others. It’s worth to take a look at these.
After Project Documentation
Since the documentation has been incrementally produced along with User Stories implementation, at the end of development cycle and roll-out of the product, the documentation - technical and functional - should describe all the features implemented.
Wrapping it all up
When deciding how much documentation will be produced, it is important to define how much documentation is just enough documentation.
Figure 2 resumes the steps proposed in this article as a roadmap to decide on documentation production.
After deciding on why, what, when and how, define the best practices to produce documentation in an agile approach, using agile software development techniques and living documentation.
Table 1 shows the output of documentation proposed for each phase.
| Pre-Project Documentation | During-Project Documentation | Post-Project (Maintenance) Documentation | |
|---|---|---|---|
| Technical | Two or three high-level architecture diagrams of your choice | Well structured code, readable by non-technical elements Test-Driven Development (BDD) Behaviour-Driven Development (BDD) Code as documentation [One User Story at the time] |
Code as technical documentation |
| Functional | Main Epics identifying the main characteristics of the product to develop | User Stories with well-defined, unambiguous acceptance criteria Acceptance criteria as documentation [One User Story at the time] |
Code as living documentation |
Conclusions
As it happens with my imaginary friends Jane and John, transitioning from traditional waterfall approach into agile approach is challenging for many teams everywhere. Questions and doubts arise from this change in the way the work has been done for many decades.
Among all the questions and doubts that arise from this change in the way teams work and especially on the mindset change it implies, documentation is a major issue, being one of the strongest changes - from documenting everything before starting to develop, to documenting almost nothing in that very same phase.
Agile does not defend in its base principles the option of having no documentation at all. But it reminds teams that the focus should always be on delivering value to the customer. In the process of producing documentation, this key principle must also be taken into account.
So, when documenting, keep in mind to document no more and no less than necessary, and when necessary.
Remind your team to document just enough and just in time.
About the Author
Starting her career as a developer and systems analyst almost two decades ago, Ester F. Gonçalves decided to explore the business side of IT on the last four years. Working as an IT Business Analyst in agile managed teams, she has become an agile enthusiast, researching on agile methodologies, on system and business analysis techniques, blogging and writing articles about it, occasionally speaking on conferences and seminars on these topics, and still aspiring to do it all much better.
