At the sixth ‘Agile on the Beach’ conference, held in Cornwall, UK, several leading practitioners of agile software delivery presented the state-of-the-art and emerging trends within this domain. Key takeaways from day one included: the value of the scientific method to drive change, and how this can be combined with storytelling for maximum effect; the value of Continuous Delivery (CD) for improving safety and speed, and that it is essential that continuous integration be implemented first; that cognitive bias must be accounted for during the design and undertaking of user testing; and how implementing CD will allow the software delivery team to learn much more rapidly in comparison with traditional delivery techniques.
Linda Rising, an independent software consultant and co-author of ‘More Fearless Change’, opened the conference, and began her presentation of ‘Better Decision-Making with Science or Stories’ by asking the audience how they had made their decision to implement ‘Agile’ methodologies:
Was your choice to implement agile [methodologies] made by carefully examining the randomised double-blind trials that demonstrate the efficacy of the practice?
I don’t think so, as there are none. Most likely your choice was influenced by the anecdotal stories of success that you heard...
Although many people consider medicine a science, Rising argued that stories have in the past driven medical practitioners to hold on to practices that were demonstrated by experimentation as ineffective. William Harvey questioned the efficacy of the long-accepted medical practice of ‘bloodletting’ and ran scientific experiments to prove his theories in 1628. However, due to the human bias for stories and anecdotes of success, it took over 300 years before clinicians fully accepted the evidence.
In modern times, when drug companies propose new medicines or techniques they are forced to run scientifically valid randomised double-blind trials, and the experiment participants are divided into three groups: action, control and placebo. Although only the action group receives the proposed treatment, some effect is often seen in the placebo group. This is believed to be due to the power of belief, the so-called ‘placebo effect’. At this point in the talk Rising asked the audience if it is possible that ‘Agile’ is a placebo?
Citing the recent teachings of Jez Humble at UC Berkeley, Rising proposed that there are small pockets of science emerging within the agile methodology space, and she encouraged the audience to utilise the scientific method when evaluating techniques and methodologies. The power of continual improvement via small iterative experiments, as championed by Teresa Amabile in her book ‘The Progress Principle: Using Small Wins to Ignite Joy, Engagement, and Creativity at Work’, should not be underestimated.
Asking questions, being open to change, and resisting the temptation to ‘cargo cult’ are highly beneficial skills. Rising proposed that stories are highly effective at transmitting information, but we should also embrace science in order to hone our knowledge. Citing ‘More Fearless Change’, which Rising co-authored with Mary Lynn Manns, it was suggested that change can be implemented within an organisation by leveraging the patterns contained within the book.
Paul Boocock, academy leader at Codeweavers Limited, presented ‘Continuously Delivering Software to Big Brands’, and stated that delivering software to production is often the most riskiest and stressful part of the SDLC. Citing the work of Jez Humble and Dave Farley, Boocock suggested the risks can be mitigated by implementing a Continuous Delivery (CD) build pipeline that assists in reducing cycle time by supporting automated practices to produce reliable, predictable and visible deployments. However, it is vital to first implement robust continuous integration (CI) practices, such as committing code regularly to a master branch within a version control system.
Boocock suggested that essential practices within CI included preventing commits when the master pipeline build is broken, ensuring all tests are run before code commit, never going home on a broken build, never commenting out failing tests, and ensuring the entire team take responsibility. The primary purpose of CD is early, rapid feedback, and as such every commit that successfully passes the build pipeline process and tests should be capable of being released into production. Some companies automatically deploy every successful commit to the master branch into production, which is known as Continuous Delivery.
Summarising the talk, Boocock stated that implementing the practice of continuous delivery for the big brands his company creates software for has enabled clients to be more involved, feedback to be provided more regularly, and work to be conducted at a sustainable pace.
In the afternoon Kat Matfield, Product Manager of Apps Inspiration and Discovery at Skyscanner, presented an informative talk entitled ‘How to Do Effective Research in an Irrational World’. Citing research by Daniel Kahneman (as documented in ‘Thinking Fast and Slow’), Matfield discussed the effects of cognitive bias on user research, and examined the inability of users participating in a study to be capable of accurately discussing or predicting the future, past and present.
Exploring Kahneman’s model for human decision-making, it was suggested that most user research focuses on when users are in a ‘system two’ mode of thought - the slower, more deliberate and more logical process - but often when users engage with a product (or the associated marketing) in the future they are frequently using ‘system one’ thinking - the fast, instinctive, and emotional mode of thought. The corollary to this is that system two cannot predict system one behaviour in the future.
The system one/two issue is further compounded by the ‘hot-cold’ empathy gap, in which people underestimate the influences of visceral drives (hunger, pain, strong emotions etc.) on their own attitudes, preferences, and behaviours. Matfield suggested that these bias and the empathy gap can be somewhat mitigated by researching and conducting the user research within the environment and state in which the user will be using the product.
Discussing the inability of users to remember the past accurately, Matfield discussed work by Caroline Jarrett ‘the approximate curve of forgetting’, and stated that it has been proven that people unknowingly make up memories if they forget. It is worth noting that the strongest and most vivid memories are the most distorted, and that human memory is not immutable - recalling a memory can overwrite the existing copy with information from the emotional and environmental state during recall. As people can’t accurately remember the past it is recommended to not ask about trivial details unless the memories are very recent, avoid leading questions, and look at recorded behaviour rather than asking for people’s memories.
When dealing with the present, the ‘social desirability bias’ can affect users ability to tell the truth because they want to “follow the herd” (even when they are alone). Matfield also presented the ‘introspection illusion’, a cognitive bias in which people wrongly think they have direct insight into the origins of their mental states, while treating others' introspections as unreliable. In certain situations, this illusion leads people to make confident but false explanations of their own behaviour, likes and dislikes. These bias can be mitigated by asking scrupulously neutral questions, not revealing the most popular or ‘correct’ answers, asking ‘what’ before ‘why’, and by treating a user's answers to ‘why’ questions as hypothesis.
The next session, ‘Testing in a Continuous Delivery World’, was presented by Wouter Lagerweij, Founder at make.io. The core premise of the talk was that although Continuous Delivery (CD) may introduce “nothing new” into the SDLC, implementing CD properly will mean that the software delivery team will fail (and learn) much quicker in comparison with traditional delivery techniques.
Wouter began the talk by discussing how the introduction of CD onto a variety of projects he has worked on forced Test-Driven Development (TDD) in order to improve the quality of the code committed and ensure a higher rate of success when passing through the build pipeline test suite. Behaviour-Driven Development (BDD) has also become highly valuable, as this not only forces the stakeholders, developers and tester (the ‘Three Amigos’) to ‘shift left’ and fully specify functionality before it is implemented, but it also guards against regression. Wouter cautioned that only limited manual (exploratory) testing should be conducted within a CD pipeline, as this is not automatable and will slow the build process. This change in approach to QA can be uncomforting to both developers and testers, as testers are used to being the safety net.
Summarising the talk, Wouter discussed the benefits of CD for him and the organisations within which he has worked:
Continuous Delivery finally allows me to do eXtreme Programming (XP) - the technical practices of agile - with my clients.
Things you could skip before [with a traditional delivery technique], you no longer can. You will fail quickly
Wouter also suggested that although the ‘programmable infrastructure’ - such as configuration management, cloud, and containers - is becoming mainstream, there is not yet sufficient tooling to allow some of the XP practices to be undertaken. This is something the industry and agile community must address. With development, test, and operations embracing agile more fully, the final piece of the puzzle is to encourage the business to embrace ‘hypothesis-driven development’. Wouter proposed that every story on a product requirements backlog should start with a well-defined business goal, around which a “cheap and quick to fail” experiment can be designed and implemented.
