At GOTO Copenhagen, Rod Johnson announced "The Software Defined Delivery Manifesto", written by multiple experts within the field of software delivery. The authors of the manifesto argue that the delivery of software "is not a detail, it is our job", and accordingly, "now is the time to engineer our delivery". Key takeaways include that software defined delivery should be core (and strategic to every organisation), well-engineered, collaborative, accelerated (through automation and reuse), and observable.
InfoQ sat down with Johnson, CEO of Atomist and creator of the Spring Framework, and discussed the motivations and creation of this public declaration. He introduced the manifesto as being written by several industry luminaries, including contributions from many others across the software delivery community. The primary goal is to encourage software developers and engineers to think about the state, benefits, and challenges of the current software delivery tooling and practices. The goal is not to "reinvent the software delivery wheel", but rather to "stand on the shoulder of giants within the automation, infrastructure as code, and continuous delivery space", in order to provide best-practices, consistency, and structure.
The manifesto begins with an acknowledgement of the impact and pervasive nature of software within the modern world, and also outlines that value is only derived from code when this is actually running within production:
We recognize that delivering useful software shapes our world. We recognize that code is the best way to specify precise action. We recognize that code is only useful when we deliver it.
The manifesto authors argue that the delivery process of software creation is not simply a detail, and accordingly, "now is the time to engineer our delivery". Acknowledging the impact that situational context has on the way that each organisation delivers software, the authors continue by stating that they "recognize that while continuous delivery is essential to meeting business needs, automating all repeated tasks is important". As delivery infrastructure is now programmable, this means that it should be defined and programmed using well-accepted practices from software development.
We accelerate our automation the same way we accelerate application development: with modern architecture and programming languages plus frameworks, libraries, and services for generic capabilities.
The remainder of the manifesto is divided into five primary sections that characterise software defined delivery:
- Core: Delivery of software is a fundamental and strategic capability for every software team and organization. Delivery code should be treated as "first class" and strategic: "Decide policy at the team and organization level; implement it with precision, without toil, in code."
- Engineered: Delivery code must be engineered to be testable and robust; not written using scripts that don't scale: "70s-era scripting languages are insufficient". Code should also be backed by a model of the software delivery domain. It should follow current good-practices within software development, such as being event-driven and easily extensible.
- Collaborative: Delivery of software requires collaboration among people and software, and between people and software. "Each person can express their expertise in code for everyone's benefit", and they should combine "best-of-breed" tools to provide collaborative automation.
- Accelerated: Repeated tasks should be automated to speed up work and avoid errors, and common functionality should be shared between developers, teams and organisations.
- Observable: There should be a common means to observe and troubleshoot what happens in the delivery process, as there would be within any a production system.
Many organisations have provided inspiration for the manifesto, and Johnson name-checked Google for their work within the Site Reliability Engineering (SRE) movement. Their approach of applying software engineering thinking and practices to the operational space, combined with the use of automation and the reduction of toil has prompted many engineering teams to attempt to recreate this within their organisation. As laudable as this is, the resulting tooling is often bespoke and takes more of a tactical approach to addressing requirements. The Software Defined Delivery Manifesto attempts to drive a more strategic approach to the entire process of software delivery.
Johnson was keen to keep separate the ideas presented within the manifesto from the implementation details, but offered his personal thoughts in relation to the work he has recently undertaken. Although Infrastructure as Code (IaC) is now a well established approach, there tends to be an over-reliance on YAML, which does not offer the power of a full programming language. As an example of tooling within in space that does follow good practice, Pulumi, with its tag line of delivering "Infrastructure as Code on any cloud with real programming languages and a consistent programming model" was mentioned.
The majority of delivery code "does not use a model", Johnson mused, which means that it is hard to reason about the process and also to re-use functionality. Citing Tasktop connect as an example of an implementation of a delivery process implementation done well, he discussed that this offering has a clear model of software delivery, and this allows the easy interchange of information between different implementations of components like issue trackers, delivery pipelines, and build artifact metadata stores.
Storing software delivery data as rich events allows an accurate history to be recorded, and may also enable additional data to be mined and added to an enhanced model at a later date. Here Johnson commented on the work of the GitHub team, and discussed the rich metadata associated within actions undertaken, such as commits, pull requests, and releases. The GitHub Developer GraphQL API v4 was provided of an example of a well-implemented API within this space, and here he liked the functionality offered by the GraphQL implementation -- in particular, the ability for a developer to query and explore the model using tools like GraphiQL, a graphical interactive in-browser GraphQL IDE.
As the GitHub model of their domain is event-based, this has easily and effectively supported the recently announced GitHub Actions, which allow developers to react to key lifecycle stages and trigger container-based actions that can be composed as part of a delivery workflow.
Finally, Johnson mentioned the benefits of ChatOps and automation, and "Why You Need a Software Delivery Machine". Discussing the recent work of the Atomist team, he has seen many engineering teams benefit from using developer-focused instant messaging tooling like Slack and Microsoft Teams to trigger automated delivery practices, such as the creation of a well-formed application template, or the deployment and execution of tests within an environment.
With a reference to the famous quote from Mark Andresson, Johnson concluded the conversation by stating that as much as "software is eating the world", he believes that "software developers are eating operations". Accordingly, now is the time to take a step back and make sure the industry is taking a strategic approach to how operational tasks can be codified and automated. He noted that "operations isn't going anywhere", and that "ops-focused skill sets will be in demand for many years to come", but much of the higher-level delivery-focused aspects of this discipline are now becoming the domain of the software engineer.
The reaction to the announcement has been generally positive online, with over 170 signatories of the manifesto at the time of publication of this news piece, although Stefan Tilkov, CEO at INNOQ, did note on Twitter that the "'70s scripting language' hate seems a little odd".
Authors of the manifesto include: Kenny Bastani, Marc Holmes, Rod Johnson, Jessica Kerr, Mik Kersten, Russ Miles, Erin Schnabel, and Matt Stine. The manifesto also acknowledges the help and refinement of many members within the software delivery community. The Software Defined Delivery Manifesto can be read in full on the supporting website.