Building Effective Developer Tools to Enable an Entire Organization to Move Faster

A key component that engineers look for when choosing to join a company is the shipping velocity. How can I as an engineer quickly take an idea in my head, get it coded up and deployed for testing and into production? As a company gets larger, the act of deploying software takes longer and longer.

Before you know it, you need multiple layers of approval and a long drawn out process for deploying even the simplest of changes.

This article demonstrates the tools that Monzo has built to enable developers, and how these tools are being used within the engineering function to deploy hundreds of times per day and beyond the engineering function to run a bank at scale.

At Monzo, we’ve found building effective tooling can help bring down the time to delivery and increase the number of changes delivered safely. Monzo is a consumer bank in the UK. Shipping features iteratively, safely and reliably is a core part of our philosophy.

Tools for staying in control

There are a lot of components needed to build a fully-featured bank. When constrained on the number of people, building tools is a really effective way to increase output. Tools are very effective at providing guardrails. Tools can be used to guide engineers towards good practices for writing quality code and enforce controls so engineers build secure systems. 

Tools can encompass a wide variety of things, such as:

• A script to automate an action; this could be something that runs locally or interfaces with other systems
• A pipeline/workflow, such as a deployment pipeline which takes care of releasing software
• A backend tool like a metrics/log store, which allows everyone to monitor and observe their system
• An editor plugin to help catch bugs as code is being written

An example of this is our deployment and monitoring systems. These are key in managing services (the typical unit of deployment at Monzo is a microservice, a singular application containing logic for a specific domain). The deployment system is responsible for building artifacts and checking whether a service has passed all the required checks and is ready for deployment. Once the checks are complete, it handles the management and rollout via Kubernetes and onto our platform. 

After deployment, our monitoring systems kick in to continuously monitor services for anomalies. All our services emit standard metrics which makes building generic alerting rules very easy.

From the engineers' perspective, they are able to go from idea to deployment quickly (everything I mentioned above is completed in under 5 minutes). Throughout this process, engineers haven’t had to learn what is going on behind the scenes in our build environment or Kubernetes infrastructure; the tooling and backend abstracts all that away. 

From an organizational perspective, the deployment tooling mandates multiple layers of checks and safeguards that you’d typically find as manual steps or part of a change approval procedure. As we learn more about failure modes and improve our code standards, we can continue to add these as automated pre-checks. 

Core capabilities also make it easy to glue integrations together. We’ve spent time building up a core library of services that make composability as simple as possible. For example, we have a microservice in our backend interacting with Slack which we use heavily at Monzo, and another microservice dealing with GitHub. We don’t need to reinvent these integrations; we have secure and defined generalized interfaces for engineers to achieve tasks like being able to post a pretty formatted message to a Slack channel across our organization or leave an automated comment on a GitHub pull request. 

Investing in developer enablement

There are two questions that are frequently asked:

1. At what stage of a company should I invest in enablement and building tools?
2. How large should a developer enablement team be?

Often, people think that enablement can only be achieved by investing in a dedicated team and giving them a mandate to solve all developer-centric problems. These become catch-all teams for things not related to core product features. Whilst it’s good to have a dedicated team, everyone across a company should be able to contribute to tooling. 

At Monzo, all engineers contribute to tooling. Some of these tools are built as part of scheduled work within a squad that stands to benefit from the time investment (such as our back-office company hub or our Security libraries). Other tools are built in an ad-hoc fashion, engineers across squads getting together and working on improving their quality of life and sharing with others (an example of this is getting Vim and Go to play efficiently with our monorepo). Ask around your organization; I would bet that various individuals or groups have little scripts to automate the boring parts. Surfacing those tools more widely and providing a framework for easily building tools is very high leverage. 

There isn’t a perfect ratio of people who should work on enablement. Many people I speak to wish they’d had tools early on. Therefore, time spent now on building tools can multiply in time saved in the future. Investing in the developer experience can potentially improve the productivity of existing engineers and help attract new engineers.

Equally important is the desire at the organizational level to facilitate automating and improving practices across a company. A core philosophy of enablement for us at Monzo is to allow engineers to self service. I see it as a hallmark of success where an engineer in a product facing team can self-serve and not need input from a Platform or Security team member because the tooling has allowed them to achieve their goals.

Not all tools need to deal with masses of complexity. Some of our tools are little handy utilities like generating a service boilerplate or interacting with an Android device during Android app development or filling out a web form for services that don’t have APIs. We’re big proponents of building small scoped tools that can be composed together with higher level scripts.

Leveraging Open Source 

Many companies have talked about the tools they’ve built and put some of them into the open source community. Many of these tools have a rich and vibrant ecosystem with multiple companies adopting and contributing back improvements.

Semgrep has rapidly become one of my favorite tools. It allows you to write static analysis checks for code across multiple languages without learning the deep implementation of a language.

This makes it easy to enforce code standards and spot bugs as part of CI. Rules are easy to define and these checks run really quickly even across large codebases. 

There’s been a rise of developer portals, especially as a means of discoverability and centralisation of service metadata. Developer portals allow engineers and everyone across the organization to see what services they own, provide documentation, see pull requests, manage deployments and much more (depending on the level of integration). 

Backstage has risen in popularity. Originally open sourced by Spotify and now part of the Cloud Native Computing Foundation (CNCF), there is now a vibrant ecosystem with many open source plugins. It’s a nice way to collate information on services, libraries, scripts and even machine learning and data models into a hub for all to discover.

When leveraging open source, it feels tempting to take a shortcut, install an open source offering and provide direct access without considering how the tool fits within an organization. I encourage everyone to think deeply how a tool fits in and what subset of functionality you want to expose. We had to do a lot of work before Backstage was introduced into the Monzo ecosystem, and we’re still continuing that journey of discovery. 

Take for example a workflow engine like Concourse. It’s easier to promote the use of a fixed set of predefined steps that are well tested and well understood rather than providing all the capabilities. The nature of open source means that the integration you were relying on (explicitly and often implicitly) may break. When you find bugs or add custom features, it’s important to be a good open-source citizen and contribute those changes back or provide feedback by engaging with the community.

Tools which enable everyone

When building tools to be used by multiple people, it’s important to minimize the barrier to adopting a tool. If you have a tool which runs as part of the command line, consider writing it in a language that gives you a statically compiled binary which makes it easy to distribute, without the need to mess around with dependency management or mismatched runtime versions. Go and Rust are popular languages for writing tools and there’s a rich library ecosystem in both languages.

Think of your tools as products that you are selling, advocating and promoting internally. As with any product, it’s important to monitor and measure how people are using the product and what their experiences are like. This can be through automated metrics (for example, how long does it take on average, and the upper percentiles to deploy using our pipeline; does a specific code check catch defects appropriately and guide engineers towards fixing them?) and through surveys and discussions with teams to understand how tools could be made better. 

As with many pieces of software, tools have a shelf-life. Providing a well supported migration path onto an alternative tool or a new system showcases a level of commitment and care, and can help reduce maintenance burdens of keeping multiple sets of tools up and running. 

What is key is spending your time embedding tools tactfully into your existing company workflow to gain as much adoption, usage, and benefit as possible.