At the Agile on the Beach 2017 conference, run in Cornwall, UK, several hundred speakers and attendees gathered to discuss the latest developments within the field of agile and post-agile software development methodologies. Key takeaways from the second day included: cultivating technical excellence within software development methodologies is vital for delivering on the promises of the Agile Manifesto; the practice of continuous delivery is a key enabler for maximising the flow of business value, but an organisation must be designed appropriately in order for this to work in an effective and sustainable manner; and although implementing security with a software system designed using a microservice architecture and deployed within container technology is challenging, there are fundamental practices that every developer and operator should use.
The second day of the conference began with a keynote from James W Grenning, which was titled "Technical Excellence, You Need It". Grenning, the founder of Wingman Software and signatory of the Agile Manifesto, began the talk by discussing how he had written embedded systems code in the 80s, and stated that delivering functionality within short cycles of continuous improvement was as vital then as it is today. Sharing insight and quotes from Demming's "Plan, Do, Check, Act (PDCA)" cycle and the practice of "Total Quality Management (TQM)", Grenning stated that a lot of the underlying principles in these techniques were behind the ideas presented within the original Agile Manifesto. However, it was hypothesised that over the intervening years the value of technical excellence and continuous learning may have become accidentally diminished, which has led to dogmatic arguments focusing only on the correct process to build software.
Grenning discussed the consensus of the ten year agile aniversary retrospective and echoed the conclusions drawn here in that the agile community must "#1, Demand technical excellence, and #2, Promote individual [change] and lead organisational change". Following Erik Dietrich's insightful blog post "How Developers Stop Learning: Rise of the Expert Beginner", we must continually learn and improve technical practices in order to avoid the trap of "a developer with ten years of experience actually having ten repetitions of a single year's experience". For example, developers should avoid doing "debug later" development, and instead focus on implementing and improving Test-Driven Development (TDD) skills. Grenning closed the keynote by discussing the importance of cultivating respect for people (something that was referred to frequently within the first day of the conference), and mused that although agile methodologies have added a lot of value for managing delivery, the importance of technical excellence must not be forgotten.
The first breakout session of the day in the business track was "Organising for Continuous Delivery", presented by Phil Parker, a partner at Equal Experts. The talk began by stating that Continuous Delivery has been implemented successfully within an organisation when "a ready feature can be put in front of real customers at little notice, and without panic", and "when speed and stability satisfy business demand". The key goal of continuous delivery is to minimise the wait time between an idea (hypotheses) of a piece of value and that value being deployed into production and being exploited by a real customer. Accordingly, this requires collaboration across an entire organisation to implement successfully. Discussing utilisation and flow, Parker stated that organisations that focus on utilisation "tend to see reduced flow and poor (effective) utilisation", and should instead focusing on flow improves utilisation. One of the ways this can be achieved is by developing in small batches.
Moving to the main theme of the talk, Parker proposed that traditional project-focused "service teams" tend to be ineffective in maximising flow, as they lack the ability to affect end-to-end change (e.g. from business idea to architecture to deployment), and this often results in increasing batch size and escalating cycle times to allow more features to be included within each delivery - i.e. the dreaded "big bang" release. Riffing off the Spotify organisational model of squads, chapters and guilds as an alternative to traditional service teams within organisation design, Parker presented a three pronged model of organisational design, consisting of teams grouped into:
- Domains (verticals): An area of business capability owned and controlled by a single group that should be able to deploy functionality independently of other domain teams
- Communities (horizontals): Loosely formed groups of people with similar skills and interests, that can collaborate across the organisation and share knowledge
- Enablers (diagonals): Teams that are responsible for building shared concerns, such as the platform or core tooling required
In closing Parker stated organisations must be aware of the impact of Conway's law and remember that although reuse is good, it comes at the cost of dependencies: boundaries between domain teams and the systems the build should be well-defined, relatively stable and not too granular.
The success of creating domain teams that have well-defined boundaries and are responsible for a (single) slice of business functionality shouldn't come as a surprise to software developers: this is simply about the architectural principles of high cohesion and loose coupling
The next talk in the software delivery track was presented by Phil Winder, founder of Winder Research and Development, and examined the role of security when developing software using the microservice architectural style and container technology for deployment and operation (the slides for this talk can be found here). Using the open source reference implementation of an microservice and container powered ecommerce "Sock Shop", a series of security concepts were discussed, and recommendations made:
- Container security:
- Set a User within a Dockerfile in order to prevent processes running as root
- Utilise hardened images (see the Center for Internet Security website), deploy containers are immutable artefacts, and configure the root container file systems to be read only
- Take advantage of Linux capabilities to limit operations that containers can undertake if compromised
- The popular container orchestration framework, Kubernetes, has excellent documentation on how to enable many of these security features
- Networking
- Implement "defense in depth" and minimise the attack surface by utilising a global firewall
- Implement network segmentation in order to prevent inter-network access
- Implement a network policy to ensure a container can only communicate with other defined containers
Winder concluded the talk by stating the security can be difficult to implement correctly and effectively. Due to the increasing spread of responsibilities across the role of architect, developer and operator, everyone within an organisation must develop knowledge of basic security principles (particularly within new technologies like containers) and become aware of the impact their work will have on the security of the entire system.
Additional information on the Agile on the Beach conference can be found on the conference website, and videos of the presentations will be uploaded on the AotB YouTube channel over the coming weeks.