Jessica Kerr, lead engineer at Atomist, presented her perspective on DevEx (Developer Experience) and how it relates to DevOps, at this year's QCon London conference. Kerr stressed the fact that embracing DevOps has led to teams taking on accrued responsibilities (to be able to own and constantly improve their systems). To reduce cognitive load on teams, we need better development tools that push down details on how systems get built and delivered, and also provide a seamless developer experience that is less dependent on how individual tools work.
InfoQ took the chance to interview Kerr and find out more about her team's motivations, the tool (Atomist) that they've built to address these challenges, and what use cases and workflows it supports.
InfoQ: You have a particular definition of what a team consists of. Could you expand on that?
Jessica Kerr: A team is everyone it takes to be successful. As a developer, "success" means delivering useful software, so everyone who is required for that is part of the team (whatever the org chart says). That includes everyone who operates the software; everyone involved in releasing it, designing it, testing it; and the people deciding what "useful" means.
InfoQ: Why do you say "DevOps is a special case of DevEx"?
Kerr: DevOps acknowledges that developers and operations people are on the same team. Then it says "let's improve the flow in this system", from packaging of artifacts to rollout and to production monitoring. With help from Continuous Integration tools, we can push out changes more quickly and more safely -- better for developers, better for users.
Building, testing, and releasing are part of the development experience. DevEx says, there's more to our experience than that! How much of it can we automate?
As small teams accrue responsibility for more and more code, we need to scale the way we work. From code maintenance to coordination, what obstacles can we smooth out next? What tools will let us automate enough work to scale ourselves?
DevOps is a great way to start working on your development experience. Don't stop there.
InfoQ: What is your personal definition of a smooth developer experience? What are its main characteristics?
Kerr: Smoothness can be hard to see, because it's the absence of pain. Here are some aims:
- When there's an error, I can see what caused it quickly.
- I can find out where each piece of data came from.
- When it's time to make a decision, all the information I need is right there.
- As the system changes in ways that I care about, I learn what happened, without a lot of irrelevant news.
- When there's something important to remember, a computer remembers it for me.
- To change the system, I can express my intention concisely, and a computer implements it thoroughly.
InfoQ: What is Atomist, what gaps does it try to address, and how?
Kerr: Atomist is a tool for automating far more of your development experience than was practical before. Just as Continuous Integration tools made it possible to run tests on every push to version control, Atomist makes it easy to respond to creation of an issue, to a request for a new repository, to test failures, to deployments, to pull request reviews -- and of course, to a code push. Our API for Software stores these events, relates them to each other, and triggers your automations to run.
Atomist offers a full software delivery machine. Evaluate each commit and lay out its path to production: format, check policies, test, package, upload, run it, verify it, approve this in Slack, deploy to prod, and see where and when it's running. Atomist is not configured with yaml or json. Your automations are code: start with examples, change them, integrate your existing tools, add what you want. Your TypeScript (or JS) code runs where you choose, and registers with our API for triggering.
Published a new version of your internal library? Upgrade every project that uses it. Want a new project created to your specification, and spun up in a test environment and then production? That's five minutes, and it's part of our starting-point delivery machine.
When a person needs to decide, they can trigger automations from Slack or in a web UI. Have a useful script to clean up test data? Wrap it in a function and make it available to your whole team as a Slack command.
The Slack integration provides another unique feature of Atomist: when an automation reaches a decision point ("the tests failed! What should I do?") it can contact a human. Post in a channel or direct message: "Tests failed on your commit." Provide the data: a link to the commit, the important piece of the log. Provide some actions: a button to restart, a button to skip tests and build the artifact anyway. The automation becomes a collaborator, and the developer is in control.
InfoQ: Why did you decide to focus on Slack? What are some advantages and challenges of that decision?
Kerr: We want to bring all the pieces of the developer process together, to incorporate information that you'd otherwise have to collect from GitHub, Jenkins, JIRA, Kubernetes. We want to bring that information to you where you are, not add yet another place to go.
Atomist's built-in Slack notification replaces dozens of spammy messages: commits, build started and stopped, deploy, review request, etc. are all correlated and presented in one message that updates, along with useful buttons like "Merge" for pull requests or "Assign" and "Label" for issues. All of these correlations are also queryable by automations, so you can create similarly rich messages of your own.
Slack messages are targeted, clear, and interactive. They are useful on all devices -- close an issue on your phone without logging in to GitHub. They let people trigger Atomist automations, and let Atomist ask people for decisions. The interactive messages are much harder to set up than a simple chatbot, so Atomist handles that for you.
Not everyone has Slack, of course. That's why Atomist also has a web workspace where you can see all these lovely correlated, actionable messages. Subscribe to the projects you're interested in.
InfoQ: How can development teams balance the "abstraction level" that Atomist experience provides with the need to understand and own their delivery toolchain in order to evolve it as delivery requirements change?
Kerr: Since all of the delivery automation in Atomist resides in code, you have full control. We provide libraries for cloning and committing to repositories, for registering with our API, and for understanding and modifying code. Our service triggers the automations, but they run in your process, and do whatever you code them to do. You're never dependent on the limited configuration options of our interface or some plugin.
One advantage of Atomist over tools like Jenkins is that you can automate every repository and team in your organization uniformly, with one code change -- or limit it, that's your choice. For instance, choose to run a linter using the rules file in each repository, or global rules of your choosing. The creator of an automation has a lot of power.
InfoQ: What kind of workflows does Atomist help facilitate? Are those purely on the "development side" or are there operational procedures that can benefit from Atomist as well?
Kerr: Oh, thanks for asking! As a developer I get very focused on my own experience, but mine is not the one that matters most.
Take a product owner or business person who has opened a ticket. You could create an automation to notify that person when a fix is deployed to the test environment, and give them link to where they can try it, and give them buttons to approve it or add more comments. The Slack interface is particularly good for this, since business people are also there.
For operations, we can choose to make Slack into a new command line, by wrapping scripts in automations and registering them with Atomist. This has a few advantages over running scripts locally: the automation developer has full control over who can run which commands; updating the automation updates everyone's use of it; and in public channels, everyone can see what's going on. When there's trouble in production and people are investigating, it's great to see what others have already tried and seen.
InfoQ: What about facilitating business decisions and visibility into the status of the software systems and on-going changes? Are there particular challenges bridging the gap between technical and business information?
Kerr: People in the larger organization do need visibility into what's going on in our teams. This is where tools like JIRA become useful: for tracking in a way that rolls up. While it's important to keep people informed, it's no fun to fill out required fields and update ticket status. If we can automate that, through automatic updates based on commits when possible and friendly Slack commands otherwise, it takes some irritating bumps out of my day.
In a sense, each team needs to implement an API to the wider organization. We can implement that manually, or automate some of it.
My goal is to gather information with zero to little cost to the busy person who has it, and provide it in the most convenient form to the person who needs it. This isn't easy, so implement a little at a time, and iterate. We'll never be able to automate every case, because it's development: everything we do is new, every change is different. As a fallback, ask a person!
InfoQ: What other recommendations do you have in terms of reducing cognitive load for development and other teams?
Kerr: Offload everything you can from your memory into the computer, in the simplest way that works for you. Spreadsheets are great. Shell scripts are great; anything you can turn from five steps to one, you'll do that thing more often. Clipboard managers are great. Slack's "/remind" feature is great; your human brain is not an alarm clock.
Ben Hammersley called smartphones our "robot brain." It makes sense now why I don't like being separated from mine. I don't feel bad about depending on my phone anymore.
If it's important, automate it. Otherwise it's too much pressure on my human brain.
The video recording of the talk will be made available on InfoQ over the coming months.