Write More, Talk Less: Building Organizational Resilience through Documentation and InnerSource

Picture this; you're new to a company, and trying to learn about the internal shared infrastructure platform. After searching the internal documentation site, you don’t find your answer, so you head to an internal messaging tool, like Slack, to ask for help.

Through Slack, you find the answer to your question from another engineer but discover that the information is not written down anywhere. This is not an ideal method of finding your answer, but it is very common. You’ll get an answer to the question, but what happens the next time someone asks the same question? This issue is prevalent with documentation, diagrams, architecture decisions and other modes of understanding why systems work the way they do.

Improving documentation practices can help build organizational resilience by making information more transparent and accessible. In my talk at QCon San Francisco 2023, I discussed the challenges of good documentation, as well as a few solutions.

Poor documentation can affect organizational resilience in a handful of situations:

• Turnover – There can be a disruption in continuity when someone leaves the company if only a single person knows how a system works.
• Onboarding challenges – In some companies, the burden of improving documentation falls on new staff. "Go through our documentation and tell us what’s missing" isn’t a very welcoming introduction.
• Re-orgs - People get moved around. Teams get moved around. Most of the time, the systems that they supported in a previous role get carried around with them. They have this baggage of support, even though they're on a new team and supposed to be building something new.
• Outages – If you must call someone in the middle of the night to find out how a system works, or find the right dashboard, your outage is going to last longer. And it’s a burden to have to call someone in the middle of the night. What if they’re on vacation?

What makes good documentation?

Documentation by itself doesn’t necessarily solve all the challenges of useful and discoverable information. Bad or overly verbose documentation can cause more problems than it solves. When writing documentation, ensure that it is:

• Useful
• Relevant
• Correct and up-to-date
• Discoverable

Documentation Challenge Solutions

There are a handful of adjustments you can make within your team or engineering organization to improve documentation, increasing resilience at the same time.

Communicate asynchronously

Take the time to think through and write down your problems before you introduce them to people. This will help you get better at communicating asynchronously.

I came across Kidlin’s Law recently:

"If you can write the problem down clearly, then the issue is half solved."

It's like rubber ducking in programming. I've seen this happen during important conversations I've had with colleagues, where we want to make some big technical decision and they haven't fully formed their thoughts around how they want to solve it. They will propose a major change in a meeting, but they haven't figured out why they want this major change – maybe they just read a blog post about using this particular tool to solve a problem. The people in the meeting will identify reasons not to make the change, and then the engineer will feel discouraged and back away from it.

I encourage people to take a day or two, and write down what the problem space is, and what the proposed solution is, and then share it with people. Let others have time to digest it, and comment asynchronously. This allows for more thoughtful solutions.

Teach and encourage writing culture

Google and other places have technical writing courses available. These can be very helpful to ensure your organization covers the basics of technical writing. This includes writing in an active voice, using correct grammar, and being informative, but not overly verbose. I took one of the Google writing courses while at a previous company with some other colleagues, and then we ran a few sessions internally. This process helped improve how we wrote as a team, and increased the amount of useful documentation we provided.

Make documentation discoverable

About 15 years ago, we had a Google Search Appliance at my job. This was a hardware appliance you bought from Google and put in your data center. It would make a Google search engine of all your documentation internally. Unfortunately, they stopped selling the product. The issue of searchability of information within companies is still a huge issue. [Image source: Barabas, CC BY-SA 3.0, via Wikimedia Commons]

While the Google Search Appliance is no longer available, there are current alternatives:

• Backstage is an open-source developer portal that was originally written by Spotify and is now a CNCF project. You can bundle all your MkDoc sites into Backstage and then search them within the Backstage portal.
• Google Cloud Search is valuable if you're using a Google suite of tools. They have a plugin to make some of the MkDoc sites searchable as well.
• Hermes is a project by HashiCorp that gives you a nice interface over the top of Google Docs and makes them more searchable.
• Github code search has gotten a lot better if you're storing all your documentation in READMEs.
• Elastic Workplace Search has a plug-in to add external sources for searchability.

Most companies are using a mixture of solutions. None of these tools are perfect, but improving the discoverability of information in your company even a little can be really valuable.

Another side topic is the discoverability of diagrams. This can be difficult, even with the best of tools. I've recently been trying to encourage people to use the C4 diagramming style, which identifies four levels of detail that you’d use in describing your codebase. While it is helpful to distinguish the level of detail, that doesn’t necessarily help with the discoverability of diagrams unless you put them all in one place. The easiest place to host would most likely be in repos where the tools are also present. Using common naming conventions and formats would also aid in that discoverability.

Image source

Create the right incentives

We don't often have an incentive structure to write documentation. It’s considered a success when you push out code or help users. It’s not considered a success to write documentation so that people don’t have to keep asking you the same question. I think our goal should be to give users as much information as needed to make them power users of the platforms. Make information digestible and curated in a way that helps them learn.

I liked the analogy from Smurti Patel’s talk based on the book Badass: Making Users Awesome, by Kathy Sierra. To make people awesome, we need to focus on the user, not the tool. Don’t focus on making a better camera, but on making a better photographer. Whatever system we have in place should be easy enough to use so that folks don’t have to ask questions. We're building complex things and that's often not the easiest path.

Understand how people work

Next, we need to be intentional about building connections and trust with people working remotely. At a previous employer, my manager had us work on team norms and stories. People discussed what their best and worst days at work looked like. Some liked going into our ticketing system and picking up a ticket that had all the details they needed right in the ticket. Others wanted more ambiguous requirements, so they could do some design and research.

As a manager, it’s important to understand how people like to approach work, rather than just getting frustrated when they aren’t doing what you expect. Being open and honest, practicing empathy, and discussing how people like to work are important to having a successful and productive team.

Provide time and space for creating

Set aside dedicated time for reading and reviewing information, and even dedicated time for writing. Try holding meetings where the main goal is to read and comment on a document; call it a writing meeting. This sounds a little counterintuitive, but it’s really useful. Meet, but leave your cameras and mics off. Read the document and add your comments to it. Being in the online meeting allows folks to go off mute quickly and ask a question if they need to; otherwise, you’re not talking back and forth, but have the focused time to read and understand the document.

It's important to understand what works best for you, your team, and your organization in creating that time and space. Some companies have identified Fridays as no-meeting days, so you can do more focused work, but Fridays aren’t always the best time to do that. Tools like Slack, which keep us constantly connected, are ruining our ability to focus, so you or your team may benefit from having set times where you disable all notifications so you can focus. Find a system that will work for you that avoids distractions.

Agree on an approach

When you get started on improving documentation, don’t worry too much about formal patterns and diagramming tools. Clear and simple documents and designs should be your goal. You can spend too much time overthinking what the right tool or format is. I’m reminded of the story where a person spent weeks finding the perfect computer to work on and the perfect editor to use. Just get started, and make adjustments from there.

If you want to start from a base that’s already proven useful, RFCs (Request for Comments) and ADRs (Architectural Design Records) are a good place to start. An example ADR template can be found below:

There are some basic steps to take when deciding on an approach:

• Brainstorm options for the approach
• Whiteboard it out, to ensure you don’t miss something
• Write up a document explaining the proposal
• Consider tradeoffs (costs and benefits) with different groups of people
• Circulate to engineers – don’t be afraid to share the document, even before it’s perfect

As a more formal option, you can use an Architecture Review Board. This is a rotating group of engineers throughout the company, changing membership every year. Whenever an RFC or an ADR is written, teams will send them to this review board, and they'll give feedback as representative engineers across the company. But it's not a mandate.

It's useful, however, to spread the information and get some feedback from others before a decision is made. Sometimes they can catch things that you wouldn't normally see, or you might get feedback from another part of the company that you may not know anything about. Especially in the platform space that I'm coming from, it's valuable in case you're not paying attention to something that might affect another group.

InnerSource (Internal Open Source)

Before I wrap up, I want to share a little information on InnerSource, which is the concept of using open-source best practices and establishing an open-source culture within your organization. You can still develop proprietary software, but you're opening up work between developers and teams internally. The key idea with InnerSource is it can help break down silos and accelerate innovation with a transparent culture.

You need to transform to a more internal sharing economy while respecting corporate culture and values, and internal organizational constraints. The idea is to drive towards openness and transparency across teams.

An easy first step is to use a single version control system and grant read access to all repositories. There shouldn’t be a reason for hiding internal code from engineers.

InnerSource can accelerate development by encouraging contributions across teams. More contributors mean that more ideas and improvements are added to shared libraries and tools. Onboarding is easier with enhanced documentation due to this increased participation. InnerSource also promotes code reuse across teams instead of duplicate projects. Centralized, shared code also means condensed knowledge and improved collaboration.

This structure builds resilience. With a broader contributor base, projects withstand turnover and reorganizations. Transitions are smoother when more engineers across the organization have an understanding of the various systems. A more robust documentation system also shortens the onboarding curve for new hires. The open culture attracts ongoing contributions that sustain momentum.

Conclusion

Documentation discoverability and transparency of information are important for any organization seeking to build resilience. Implementing strong documentation practices and shifting cultural norms to favor knowledge sharing is essential work. Some tactical solutions, like better asynchronous communication, incentives aligned with writing, and structured formats, can move teams toward resiliency. InnerSource practices can also help and companies can adopt what works for them.