Transcript
Grizzanti: My name is Dave Grizzanti. I'm going to talk a little bit about documentation and InnerSource. Before I do a more formal intro, just to show a quick example of some of the things I think we deal with very often. Let's say you started at a new company, and you're looking to get support for a shared infrastructure platform. John here goes to a Slack channel that he thinks is the right place to ask for help. He says, I'm new to the company, and I'm looking to set up a new app on our shared infra.
I ran over to the docs, but I didn't find the answer to my question, is this the right place to ask? Another helpful engineer on the support team says, John, yes, this is the right place to ask. What do you need help with? John says he's looking to set up a custom DNS record for an app he's launching on the platform. The other engineer says, sure, let me dig a few examples for you. This is a common use case I see a lot. Why this isn't documented, could be lots of reasons.
Let's say John is on the team or at the company for a little bit longer, and he wants to ask his team for the app that we have running in production. I'm trying to understand how traffic is routed from the internet through our network to the app. Do we have any diagrams that show that so I can take a look? An engineer on his team says, "I'm not sure we have an exact diagram that shows that, but I drew something for the CTO a few years ago for a presentation. Let me see if I can find those diagrams." How many of you face similar situations to this or have said these things?
That example about the diagram, I think I've said that about three times in the past, maybe not for the CTO. I think these examples are really common in our industry. I see them all the time. I think they show a little bit of a crack in the way that we deal with information sharing and support. A lot of times we handle this with more formal processes, like file a Jira ticket to get support, somebody will take a look at it. That's not a great way of doing it either. No one wants to file a Jira ticket. I certainly don't want to do that. Also, I think it discourages people from asking for help, just because there's too much process in the way.
I'm a Principal Engineer at The New York Times. I focus on developer productivity and platform engineering at the Times. I think a lot about these use cases where people are asking for help for the platforms we're building, and how to get them to move faster, build more paved paths to let them get to what they want to do, which is build products. Today we're really talking about building organizational resilience. I'll weave in some of those ideas as I go through this. Really, I want to talk about how some of these documentation examples and open source concepts can help build resilience for organizations.
Information Availability
I think a lot about developer productivity, like I mentioned, and how to engage with engineers to make their lives easier. Part of this process is making documentation easier to find and understand. I think we just have a sea of information now. We're overwhelmed with documentation that exists that's either out of date, or has correct information but not exactly what people are looking for. It's often easier for somebody to ask just for help on Slack like I showed, or whatever communication tool you might be using, than reading through pages of documentation and trying to find the answer themselves, or piece together a solution from a bunch of different places.
The issue with the Slack approach, I think, is that we often answer the questions but we don't go back and make that content available for others. It repeats that process over again, like we answer the question, but we don't take what we answered and make it available for other people. This doesn't just stop at documentation. It extends to diagrams, like in that example I mentioned, architecture decisions and why systems work the way that they do.
You can also argue that this happens with software too. Large companies have lots of duplicative software across teams that maybe do the same thing. Teams often maybe write something that already exists, because they don't know that those things exist in the other companies or they're available for sharing. To sum that all up, I think my thesis is information either isn't available or discoverable. I'm going to talk a little bit about how maybe we can solve that.
Effects on Organizational Resilience
I want to talk about some of the effects of this bad documentation or information unavailability and the effects on organizational resilience. The first thing is turnover, so folks leaving the company. This is a big one I've seen cause harm to teams and disrupt continuity, where a single person or even two people might know how a system works. When they're gone, or they're on vacation, somebody asking me for help like that on Slack means that somebody can't get an answer. This might mean that you can't move as fast as you want, or you can't get the help you want until a new person comes up to speed or a new person is hired. The next is onboarding challenges.
I know for myself, because I started at the Times about a year ago, one of the tasks I was given was, can you go through our documentation, try to do this and tell us what's wrong. Which is interesting, if you like fixing documentation stuff, but some new folks might get frustrated that now they can't onboard very quickly because they're in charge of finding the holes in the documentation. You're putting this burden on them to improve. Either way, it's not a great experience. Either people can't get up to speed quickly, or they're frustrated by the lack of documents.
The next thing is reorgs. For any folks who've worked at large companies, this is pretty common. People get moved around. Teams get moved around. Most of the time, what happens is the systems that they supported previously, gets carried around with them. They have this baggage of support, even though they're on a new team and supposed to be building something new.
Either because they didn't have the time to document or draw the right things, or they just didn't have the right practices in place. Then the next thing is outages. In the middle of the night, if you have to call somebody and wake them up to find out how a system works, or you can't find the right dashboard, or find the right documents, or something like you're on vacation again, now your outage is going to last longer.
How Can We Improve?
What are some ways that we can improve this? The first thing I want to talk about is documentation. How to create an environment where good documentation practices are encouraged. Ways of making your documentation more discoverable, and teaching people how to approach documentation and maybe how to write technically, in a positive way.
Then the second thing is this concept of InnerSource. This is the idea of doing open source internally. How to create projects that are sustainable, internally. Get more contributors across the company, instead of your main team. Spend less time building the same things, and make happy customers and build trust among engineers.
Documentation
Let's talk about documentation first. What makes documentation good? By itself, documentation doesn't necessarily solve a problem. A couple key things, I think, is that it needs to be useful. It needs to be relevant, correct, and up to date, and discoverable. Documentation is not just words. It could be diagrams. It could be information in READMEs. Or it could just be code comments. I think we've rode the wave of whether or not code comments are good or bad, or you should write code that's self-documenting. People don't need code comments.
Oftentimes, people will write very complex things in software, and they don't necessarily know where to put the information about it. I think code comments have their own value in this documentation story. Bad documentation often causes more problems than it solves. If you have really long verbose documentation that doesn't really answer a question, somebody spent all this time writing this, curating it, and if no one's using it or reading it, is that really any more valuable than not having anything?
The Challenges to Making Good Documentation
Let's talk about some of the challenges to making "good documentation." What I found, I think, is that people often want curated and custom answers to questions, going back to that Slack answer I saw before. Either they can't find the information in docs or they don't want to read, so they come to ask you a question. This is maybe like a slightly contrived example. I wanted to promote The New York Times cooking app while I could.
Let's say you were on the Times cooking app and you wanted to make a recipe, and you're like, "Ok, I'm going to be adventurous. I've never made bulgogi before. Let me make this." I'm looking at the list of ingredients, and I say, I don't know what gochujang is, I can look this up. I don't know if anybody's ever used the cooking app, but reading the notes is a very fun rabbit hole to go down because people ask the strangest questions and also suggest the strangest substitutions. One of the comments I actually saw on here was, what is gochujang? Very fair question. To me, I was like, why would this person ask it in the notes? Why would you not just Google it? I find that people often, they want an expert answer.
Maybe the cooking community has a better interpretation about what this is than Google does. Maybe they don't trust Google. To me, it hits on like this idea of people wanting someone to give them a specific answer to their question. They don't necessarily want to look it up or trust somebody else. If you haven't looked at the notes, or looked at this Instagram account for The New York Times cooking notes, I highly encourage to go take a look.
Another thing is, communication face-to-face, over writing or reading. I've been remote. It's different than in the office. I find people often will DM me and ask me for help with something like, can we jump on a quick call, and let's talk this over? Maybe that's ok in the beginning, as long as something comes out of it.
When you're talking over a problem or situation that's like in a support nature, and you come up with a solution, that's not written down anywhere, it's not even on Slack. That cycle of not producing anything of value afterwards continues. There's a place for helping people in a one-on-one or talking through a problem. I think oftentimes, if nothing's produced out of it, or this is your cycle of always answering questions that way, that can become a problem.
The next thing is, writing well takes time and focus. I'm not sure how much this resonates with people. This is what my schedule looks like very often. I don't know how anybody finds time to focus if your schedule looks like this, especially with the chaotic nature of the meeting blocks. If you ask, I find often that when I see somebody answer a question on Slack, or help someone, I say, can you take that information and go put it up somewhere. Synthesize what they asked, and your response, and put it up on our docs.
Most of the time, they don't do it, or they do something and the result is not super helpful. Often, it's because I think, again, we're just too busy to find blocks of time where you can focus a lot. This may not be the case for all engineers. I think I found that in a remote culture, a lot of times, a lot of these blocks are dedicated one-on-ones that people have with people, or just meetings that are recurring, to check in on status. Everybody needs these meetings to feel like they're connecting with their team, and it doesn't necessarily help with the focus time.
Solutions to Documentation Challenges
A few solutions. We really need to get better at communicating asynchronously. I think the idea of communicating more but less frequently, is something that I've been trying to do more of. Meaning, put out more information, but don't feel like you need to talk with somebody every day. I came across this quote in this law the other day that I thought was interesting, which is the idea is if you can write down the problem clearly, then your issue is half solved. It's referred to as Kidlin's Law.
The idea is like writing down a problem, getting it out of your head helps clarify and work through your thoughts. To me, it's like rubberducking in programming. Like, if you jump to your first thought, or thinking about something on the surface, and throw on an idea, oftentimes, it'll die on the vine. I've seen this happen with important conversations I've had with colleagues recently, where we want to make some big technical decision and they haven't really fully formed their thoughts around how they want to solve it. They will propose it in a meeting.
Like, I want to use Helm charts for Kubernetes versus some other tool, which is very controversial, but they haven't really figured out why they wanted to use it. They may have just read a blog post about using this to solve some problem. The mass or the group will throw a bunch of negative opinions at them. Then they feel discouraged and back away from it.
Where I've been trying to encourage people to say, ok, take a day or two days. Write down what you think the problem space is, what your solution to that is, and then share that out with people. Let them comment asynchronously, let them take time to digest it, and then we'll go over it with everyone. Taking the time to think through and write down your problems before you introduce them to people.
The next thing is, teach and encourage a culture of writing. This is something that I started doing in my last role, which is this idea of, people have different writing styles, all that doesn't necessarily lend itself well to technical writing. Google has these technical writing courses, that they teach technical writing, and some of it is grammar. Some of it is just how to write in a way that's verbose enough, but clear when you need a particular technical point.
Like, don't write in passive voice, write in active voice. They run one or two classes that you can join. They also give you the materials if you'd like to run them internally. I joined one of them maybe two years ago, and there was like 60 people on the class, so it wasn't all that productive with that many people. We took the information and ran a few sessions internally, which were successful. I wasn't there long enough to continue doing that. It's something that I've thought about bringing it back to the Times.
The next is, make it discoverable. I had this at a job, 15 years ago. It's a Google Search Appliance. It's something that you could buy from Google and stick in your data center. It would essentially make a Google search engine of all your documentation internally. Unfortunately, they killed the product at some point. I think that this issue of searchability of information in companies is a huge issue. I've never seen a great solution for it.
A few things that I've seen recently, and some things that we're doing at the Times now, that are some alternatives, that other people are familiar with these tools. I'll go through them and talk about pros and cons. Backstage is an open source developer portal that was originally written by Spotify. It's now a CNCF project. They have this concept of TechDocs.
You can bundle all your MkDoc sites into Backstage and then search them within the Backstage portal. If you happen to have a bunch of MkDoc sites that are all spread across your company, you can put them into Backstage and make all the docs searchable. That works well for MkDocs, maybe not so well for other things. At least technical documentation is all searchable in one spot.
Google Cloud Search is actually really valuable if you're using a Google suite of tools. That's what I use most often now. They even have a plugin thing that you can integrate, I think with MkDocs to make some of the MkDoc sites searchable as well. Hermes is a project by HashiCorp that gives you a nice interface over the top of Google Docs, that also makes them more searchable, which is nice.
GitHub code search has gotten a lot better, if you're storing all your documentation in READMEs. Then something I've looked up and not used a ton is Elastic Workplace Search. I think this is a pay feature. If you're an Elastic user, you can plug in all these external sources, and then make it searchable, and Elasticsearch will index it for you. A couple of things to solve this searchability problem. What I've seen at most companies is they're all using slightly different flavors of all of Google Docs, Markdown, things in GitHub, so it's not perfect. If you can focus on one documentation type, it can be really valuable.
The next is, create the right incentives. This is something my boss said to me the other day when I was complaining about trying to get folks to write stuff down, is that we don't often have the good incentive structure to write documentation or to write stuff down. The incentive is to push out code or help users. If you help them get something done, or ship a product, like that's the success story. Not, I'll go back so the next time somebody doesn't need to ask the question.
I think we need to reward people for working on this stuff, just like working on coder solutions. Smruti mentioned in her talk on platform engineering about this book, by Kathy Sierra called "Badass." The quote from her talk, and from the book is about making better photographers not a better camera. That idea really stuck with me because we oftentimes think about making our platforms better or good enough so people don't have to ask questions. It should be really easy to use they don't have to ask questions. I think we're building such complex things that that's not often the easiest path. I think our goal should be to give users as much information as knowledge to make them power users of the platforms. Making information digestible and curated in a way that helps them learn.
The next is, understand and learn how folks like to work. I think getting to know each other really at a personal level helps. There's lots of different ways to do that. Especially in a remote culture, it's challenging. I think you need to be intentional about building connections and trust with remote people. Oftentimes, we ask people to do things like, why didn't you just write this down? Or why didn't you do that? Some folks may feel intimidated about writing large documents.
They need more focus time. I just think understanding that relationship between people and between your coworkers is important. I think practicing empathy in that space is especially important. One of the things my manager did at a previous employer, was we did this team norms and stories, where we got all in the conference room for a few hours, and people went through what's your best and worst day, not personally, but at work.
Someone said my perfect day is, "I go into Jira, and there's a ticket I can pick up, and it has all the tasks I need to do. I can pick it up and work on it, and then I can move it to done." I don't want to think about all the complexities of like, what I need to do, I just want to be given the list and I want to work on it.
Someone else may be like, "I want more ambiguous requirements. I'd like to like design this myself, and then work on it." I think until you get that information out of people, it's hard to assume how you should approach things with people. As a manager, you might be getting frustrated like, why isn't this engineer working on what I asked them to do? Maybe the way that they like to work is not something that you understand. I think being open and honest, and just discussing how people like to work is important to having a successful and productive team.
Then give the time and space for creating. Maybe you have dedicated time for reading and review, dedicated focus slots for writing, and writing meetings. That sounds counterintuitive. A few times we've done this like, let's just schedule a block of time to block it out. We'll put our cameras off and turn our mics off, and we'll just sit there and read this document and then comment on it. I've done that a lot recently for proposals that we've been writing for standards at the Times, where we have a scheduled block of time, and we'll review some of the documents that have been written, to take back comments. Being online, being on the meeting allows us to just go off mute quickly and ask a question if we need to, otherwise, we're not talking back and forth.
Lastly, let's talk about just agreeing on approach, to writing and formulating some of these documents. Don't worry too much at first about formal patterns and diagramming tools. I think people sometimes get hung up on like, what's the right format. You spend two days searching on Google for the perfect template, or whatever you're going to use.
Reminds me of the story somebody talked about, where they spent three weeks finding the perfect computer to work on and the perfect editor to use. You can spend too much time overthinking what's the right thing. Don't worry too much about formality, just clear and simple documents and design should be your goal. If you do want to decide on a style or a format, RFCs and ADRs are a good format to choose. They have structure to them. Most of the time you can find a lot of information online about companies that follow these approaches.
If you struggle for where to start, and you want something to use, these are a good basis. This is an example of an ADR template that a colleague of mine at the Times, Indu, proposed for her team to start using. It's not super complicated, just a question to decide on, what's the context, what's the recommended decision, supporting arguments, those sorts of things? Just gives you a place to start and a place to write down your thoughts, so you're not just staring at a blank Google Doc and being like, how do I ever write down this long document? Especially if you've never done one before.
Some basic steps. Get together with your team, brainstorm and approach. Maybe do some whiteboarding. Write up the document, whatever format you choose. Consider tradeoffs. Then, circulate to engineers. I think one of the also common things I see is that people are afraid to share the document before it's perfect. I think circulating it, and getting feedback early and often is important. If you find yourself wanting some more formal approach to this, I've seen a mix of these things work.
At the Times, we have this thing called the Architecture Review Board. That's not like an ivory tower architects' person, it's a rotating mix of engineers throughout the company, that changes every year. Whenever an RFC or an ADR is written, teams will send them to this review board, and they'll give feedback as a more slice of engineers across the company. It's not a mandate.
You don't have to go to them before you launch something. It's more of a nice to spread the information around with everybody to get some feedback before a decision is made. Sometimes they can catch things that you wouldn't normally see, or you might get feedback from another part of the company that you may not know anything about. Especially in the platform space that I'm coming from, it's valuable in case you're not really paying attention to something that might affect them.
InnerSource
Let's wrap up on documentation and talk about InnerSource, which is the second topic, which focuses more on software sharing, and a bit on documentation within the scope of these projects. What is InnerSource? InnerSource is internal open source. It's the use of open source best practices and the establishment of an open source culture within your organization. You can still develop proprietary software, but you're opening up work between developers and teams internally.
I think the key idea with InnerSource is it can help break down silos, and accelerate innovation with a transparent culture, like open source. Saying, just do open source in your company, is not necessarily easy or quick. It's definitely a journey. You need to transform to a more internal sharing economy, respecting corporate culture and values, and internal organizational constraints.
The idea is to drive towards openness and transparency across teams. I think an easy one is, use a single version control system, and everyone should have read to all repositories, which I think is a common practice in some orgs. I've seen that in previous companies that's not always true. There shouldn't be a reason why you're hiding code from internal engineers. Make sure everybody at least has read.
Benefits of InnerSource
A couple areas, I think, where InnerSource can help. If you manage to get this up and running, you can definitely develop software faster by having more folks helping you, even if it's just one person from an outside team, who has ideas on a shared library or shared tooling. Improving documentation. The onboarding example I mentioned a while ago, if you don't have a lot of new engineers starting out on your team, if you have new people coming in often that want to contribute or want to help, they're constantly looking at your documentation, looking at your contributor guide, they can help improve docs for themselves, and also for the wider company.
Reuse code across teams. Maybe you have people rewriting the same tools across the company that do the same things, especially in large organizations, so build a common set of shared libraries that multiple people can contribute to. This also builds trusts and improves collaboration: engineers knowing each other across teams, helping each other, those sorts of things. I've seen the develop software faster and the reusing be really useful with maybe internal APIs or internal platforms with supporting tools.
We had a pretty successful DNS tool at my last company, where we had lots of people from outside the team build libraries, and also like Terraform modules, those sorts of things, that weren't developed by the core team, they were developed by developers that were using the APIs. It was an example of developing software faster and helping this trust among engineers.
How Does InnerSource Improve Resilience?
How does this help with resilience? I think from an employee turnover perspective, you don't only have one person that understands how the software works, or even one team, because you have people from across the company contributing to it. Maybe they can help take over. They can move teams, if you have staffing issues.
Helps with onboarding time, because of the documentation that I mentioned earlier. Also, with more people contributing, there's just more activities, so I think new people coming on can get started quicker. Then the same thing with reorgs. If you have teams moving, if software is getting handed off to different teams or moved around, the more people that understand how things work and are contributing, the quicker you can get started.
InnerSource Commons - Patterns
InnerSource Commons was founded in 2015, and is a nonprofit. It's the world's largest community of InnerSource practitioners. They're dedicated to championing this idea of InnerSource and building a community around it. I've been working with them over the last few years, and contributing to this thing they call patterns. I want to go over a couple patterns that I think are valuable and can give you an idea into how to get started with maybe promoting InnerSource inside your company.
This is a mind map tree of all the patterns that they support and a couple categories that they fit into, so begin, adopt, grow, and scale. The idea is that you can start at begin, and travel down depending on where you are in this journey. I'll talk about a few of them across the four categories. Let's say you want to get started, but applying open source practices doesn't work in your company when maybe some folks are lacking an open source background.
The pattern that they have for that is called documenting guiding principles. The idea with documenting guiding principles is it provides clarity on purpose and principles to users. Why does the organization want to adopt InnerSource? Which InnerSource principles will help address these challenges?
The next one is, new contributors have a hard time figuring out who maintains the project, what to work on, or how to contribute. The pattern there is standard based documentation. This is the idea, which you've probably seen a lot in open source projects. It addresses the need for clear documentation, so maybe a README, contributing guide.
In the open source world, that also includes things like licenses, which may be applicable internally for you as well. It enables a more self-service process for new contributors. They're not pinging you on Slack and asking you, "I want to fix this bug, is this the right way to go about building the application?" I've seen this example, actually fairly recently. We've collected a couple people across our platform engineering org to help work on a few things, and it took them two to three weeks to add a feature that was very basic, because they didn't know how to test what they were working on.
Not just unit tests, but test it as the integration point across systems. I met with them. We worked through it. We improved some of the documentation. Then they opened the PR and the team that wrote the app changed the way something worked, so now all their tests weren't working anymore. I didn't know that either. I had to go back and ask them. They didn't update the documentation. I think just being in this habit of maintaining your contributing guide, even if you're the only team working on it. If someone else wants to contribute, or another team member in a sister or a brother team wants to contribute, it's valuable to keep this stuff up to date.
Next is, teams may hesitate to adopt a project, if they're unsure of its maturity. I think this is a pretty standard thing in the open source world where if you go look at a popular open source project, most of the time they have release notes, they have an available binary that you can download, or brew install, or whatever. I think internally, we don't often publish standard artifacts. This pattern is a standard release process, which is publishing release notes and usable artifacts.
These practices show dedication to the project. It demonstrates commitment to sustainable software, and increases confidence when you're publishing a quality product. For internal maybe like CLIs or binaries, oftentimes, if the team uses them, they'll just build it themselves locally and have it on their machine. They don't make it available for other people to install easily, or like publish the release on their internal GitHub. I think if you want other people to be able to use it easily, you need to make it available and also show them that you push out releases every month and are publishing release notes, so they know what to expect.
The next is, thank outside contributors for their time and effort. This is, praise participants. I'm not sure how common this is in open source. I feel like I've had mixed reactions to this for people. The idea is that, it feels good for people to be recognized, even if it's something silly. Sending somebody a thank you note, sending them stickers if it's an open source project.
I think increased recognition is an avenue to influence and growth. I think if you thank people, they'll be more likely to contribute a second time. They'll feel good about making something better, versus their contribution just being ignored. I think everybody likes swag. It's a valuable thing to give away. Having projects internally in your company, think about branding them. This is something I've been trying to bring to the company I'm at now where we had a project that had a name, and we were having an onsite at the company where we were going to be talking about the project.
I said, we really should print stickers and give them out when people come to the talk. They're like, we don't have a logo. I was like, let's just make up something on Photoshop, and then print it out and send it to them. We did that. Then people took the stickers and put them on their water bottles, and the name becomes more synonymous with the project, because people have this little token.
Next thing is, potential contributors cannot easily discover projects that interest them. This is the idea of having an InnerSource portal. This could be like a website you build yourself. It could be tags you use on GitHub. It's just this idea that you want to make the projects discoverable in some central place. Backstage has a plugin called bazaar that is an attempt to make this InnerSource portal idea.
You can list all the projects in maybe your GitHub repository that meet some qualifications for InnerSource, whether they're ones that you're trying to promote, or that have a good contributor guide or whatever. Some curated list, that you're telling people, so they're not just trying to figure it out themselves by wandering around GitHub.
Create participatory systems throughout your software lifecycle. This goes back to that RFC idea I mentioned before, but cross-team decisions through RFCs is important. Again, don't focus too much on the format. Publishing these kinds of standardized format documents out in the open through that Architecture Review Board, or just through some engineering email list allows for discussions early on in the design process, and increases the chances to build solutions with a high degree of commitment from all parties. People see that you're going to build this new tool or make changes to the platform. It's not a surprise to them when it happens, because this document's been circulated more widely.
Key Themes
Some key themes that I mentioned here is, documentation needs to be accurate and discoverable. Find what styles and formats work for you through trial and error. Don't assume that if somebody else is doing one thing it's going to work for you. Make the time and tools available to contribute successfully. The incentives I mentioned before, just giving people the time to focus and write tough stuff down.
Build a culture of InnerSource for key projects and platforms, and follow some of the prescribed patterns that I mentioned. The community is always looking for new patterns, and also new companies to sign on to patterns they have. It's important for them to see examples in the wild of these patterns in practice.
Takeaways
Discover how your team likes to work. That idea of sharing and going through what people's ideals day is, I think is a good way to understand what people want out of their work environment. Make dedicated time and space for reading and writing. Investigate the patterns I mentioned.
Questions and Answers
Participant 1: On the topic of InnerSource, it strikes me that monorepos get at some of the same sorts of things. Can you talk about how monorepos and InnerSource might or might not be able to do that?
Grizzanti: The Times has a few monorepos. The news app I think is a monorepo. I think it does help with some of the information fracturing across repos. Like all the code is in one spot, you might only have one README, one place to put a Dockerfile or a Makefile to do builds. I don't think InnerSource has a pattern on monorepos.
That's actually an interesting thing to chat with them about. I do think it has pros and cons. I've seen some of their build pipelines are a little gnarly, because they're trying to like, if you check something into this directory, how does that affect the build pipeline? I think from centralizing all the information in one spot, that can definitely be helpful.
Participant 2: Can you go through maybe a few more practical ways to get time and space made to be able to do much more, for instance, communications?
Grizzanti: I think there's two aspects there. From a personal perspective, you can block off your calendar. Whether or not that works super well for everyone in every organization is organizational dependent. I think also, you need to think about like, when do you work well on those sites of things? Being introspective about like, am I better in the morning, am I better at night? Where would I prefer to focus time, and advocating for that.
Some organizations, we toy with this idea of having a focus day where there's no meetings. It was on Fridays, which I think is probably the worst day to do it. Because I feel like at the end of the week, people were like, I don't want to spend all day working on documentation on a Friday. That may not be successful. Maybe you just want to wrap up your week and get a bunch of other stuff done. For me, the best time to do it is in the morning. Luckily, even though The New York Times is based in New York, I work with mostly West Coast people, so I don't have any meetings until like 12:00 or 1:00.
Most of the time in the morning, that works really well for me. That doesn't work well for everyone, though. I think there's a balance of like doing the blocked calendar thing, and also figuring out what times work for you and advocating with management. We can't just expect people to be able to spend all this time reading huge documents that we're putting out, commenting on them and writing their own if we're in meetings all the time. I know we've been talking a lot about making more space for that, trying to cancel more meetings, doing stuff more asynchronously.
I really like Cal Newport's book, "Deep Work." Also, there's another one called, The Death of Email. He talks all about how tools like Slack, and just the constant connectedness is ruining our ability to focus. He goes into a bunch of styles for folks to do more focus work, and figuring out what works for you. Like investing time and money into something may help you focus because you're being more intentional about it.
He tells a story in the book where somebody was trying to write a book and he never had any time, so he bought a flight from New York to Japan, and then got off the plane and then flew back. All he did on the plane was write. It was really useful for him because he couldn't do anything else. He spent all this time and money doing it, so he felt the need to focus. Obviously, that's a little impractical for most people. Maybe you need a different place to go to write instead of just being at your desk. Toy with those ideas and figure out what works for you.
Participant 3: In my organization, [inaudible 00:43:19] we have many country sections, many viewpoints to diagrams, many [inaudible 00:43:29] spread across many different systems. I was wondering if you have any tips or suggestions for how to organize that, and encourage other people to keep them organized?
Grizzanti: I think we have similar problems. I think the diagramming for us is a bigger problem, because everyone uses a slightly different format, it's at a slightly different level or scale. This doesn't help with discoverability, but I've recently been trying to encourage people to use the C4 diagramming style. It's like, this idea that whenever you're drawing something, there's four levels.
The example he uses is Google Maps, you're looking at it from the continent view, or you're looking at it from the country view, to the state view, to a specific road. At least that gives people a common like, these are the levels I should aim for to keep the formats common. I don't think there's a great way to search for diagrams, though. At least I haven't found anything to make discoverability of diagrams be easy unless you put them all in one place. I think the easiest place to probably host those is in the GitHub repos where the tools are also present.
As long as they have common formats, they may have common endings, and you can use code search to find those things. I think storing them in the tools that you're drawing them in, and keeping them there only, and producing PNGs or JPEGs, and putting them on a wiki doesn't really help. I think keeping the original sources and the copies of them with the source code repositories, is probably the best thing that I've seen, because at least then when you're looking at the actual repository, the diagrams are cohosted with them.
Participant 4: I'm curious about one of the earlier points you had, that people are looking for curated specific answers. That is something that I see fairly often. I think it's almost like a cultural thing. I'm wondering if you have any tips for dealing with that.
Grizzanti: What we've tried to do is when people ask questions that we know are documented, to encourage them to go look at the documentation, like not RTFM. Don't be mean about it. Like, "We have this documented on this place, go read it." One thing I've seen work well is using Slack bots to try to understand what people are asking, and then point them at the docs. That requires a lot of investment, though, and it's per team.
I think encouraging the people who are answering the questions to understand that this issue exists, and if you just answer the people's questions every time, then they're going to keep coming back and asking over again. If you look at their questions, point them at the docs, or after you answer it, write the documentation up and then point the next person back at that. I think that's the best we can do at least for now. I don't think this problem is unique to our industry either. I do think that a lot of people are just used to getting curated answers to things, the cooking app being the example.
Participant 5: I was just wondering if you had any recommendations for keeping code [inaudible 00:47:13]. Then, also, if you've had any success with tools that generate documentation, like Javadocs, or Doxygen, or something like that?
Grizzanti: There was a movement around continuous documentation, which is this idea of keeping your docs with your code and treating them like source code so that they're updated when the code is updated. Whether that's like, you have some CI that runs with your code that says like, did you update the documentation, or reminders?
I think is really the best process we can do. I think that's also very cultural, to just stay top of mind of like, "Ok, I changed something. Do I need to check if the documentation should be updated, or the diagram should be updated?" Most likely if you're changing something, there's likely something that needs to be updated somewhere. Just seeing that often in pull requests can build that culture.
I've seen that work for APIs, like Swagger, and Javadocs, and Godocs. I feel like oftentimes, people just to get around the Go complaining, you just put one line at the top of the functional. It's a balance. The self-generating stuff is useful. A lot of times, it's just boilerplate and it may not tell you all the complexities of the stuff you're writing. I think that's also just when you're doing pull requests, reviewing, make sure people are actually documenting what's happening, if it's valuable.
I was talking to a former colleague about Ruby. I remember when I was writing Ruby 10 years ago, rubocop was very militant about keeping your functions very short, having certain number of lines. I think that's something that we don't often check. Really long functions that have no documentation are not good. If you keep them like four or five lines, maybe they're self-documenting enough that you don't need a lot of code comments.
Participant 6: [inaudible 00:49:33]
Grizzanti: I did think of one other tactic I've used in the past to improve documentation is to do documentation sprints. We did that recently when we were launching the GA of our platform. Maybe every couple months, maybe once a quarter, you say like, we're not going to work on any features for the next two weeks. We're just going to work on improving docs. Maybe once a quarter is too much, maybe it's not often enough, but being intentional about fixing that, and improving it and making it better is valuable.
See more presentations with transcripts
