Advent of Technical Writing: Holistic Documentation Reviews

This is the nineteenth post in the Advent of Technical Writing series, wherein I will share something I have learned from my experience as a technical writer. My experience is primarily in software technical writing, but what you read may apply to different fields, too. View all posts in the series.

In your day-to-day life as a technical writer, you will make a lot of incremental improvements to documentation. This might involve adding a new guide to a larger documentation site, adding a new section to an existing post, updating the navigation structure for a product, updating screenshots, and more.

Every so often, I recommend taking a step back from the projects on which you are working to ask: is this documentation, as a whole, achieving its goals?

This is a question I asked myself earlier this year when working with the Roboflow product documentation. The documentation was complete and comprehensive, but we had built up some "documentation debt" per se. For example, some screenshots were out of date. Some sections were not ordered as intuitively as they could be for new users.

My analysis, and the growing general sentiment that we could improve our documentation, led to a holistic documentation review.

Committing to a review

In my day to day work, I started to see improvements that we could make to our documentation. These improvements would help us better position our products with our new products in mind, aid in users finding the information they need, among other things.

If you think documentation could be better, start by flagging it to the relevant party (product owner, technical marketing manager, etc.) that you see opportunities to improve a documentation site. Reference specific instances where you think the site could be improved. Have a conversation. It may be the case that your manager and potentially other people in the organization feel the same way about a review as you. If there is interest in doing a holistic review, share that you are interested in taking lead on the project.

What to ask, and how

We started the Roboflow documentation review with two main questions:

• What are we doing well that we should do more of?
• What could be improved?

I started by writing a document that listed suggestions, and then worked with the team to facilitate feedback. It is important that you not only ask the right questions, but that the right people are involved. In the case of this project, we invited the whole company to participate.

Indeed, documentation is something in which every team has an interest. Accurate, comprehensive documentation helps customer support have better resoucres to which they can point users. The product team can conifdently state that there is documentation on how to use what they have built. The sales team can confidently refer to documentation that shows how to use our product to solve a problem in customer calls.

We collected feedback for a few days, as well as ideas on how we could improve our documentation. I proactively invited people to participate, with a Google Doc in which suggestions were noted. I invited comments on existing feedback, too. The more input, the better. Getting people who work directly with customers involved on a regular basis was important to me. They could note common questions they had heard that could be used to inform improvements.

I set a specific date after which feedback would be collated and implemented. I was always open to more feedback, but setting a deadline helped me ensure that people knew by when I expected input.

Here were some of our findings:

1. The structure of navigation links was not linear, which made it hard to find some information;
2. Some documentation could be simplified;
3. Text was styled inconsistently across some pages (i.e. some pages used bullet points liberally);
4. We had new products that we wanted to ensure were visible, among other findings

At this stage, we knew both that we could improve our documentation and had an incline as to areas for improvement. Some people left excellent ideas that we could use to inform tactical changes, too.

But there was one thing we needed to do before making significant changes: user research.

User research for documentation reviews

Internal stakeholders have extensive knowledge that is useful in preparing documentation. For example, product owners know about the features available in a product and how to use them. With that said, there are many things that internal stakeholders cannot tell you because they work with your product or documentation regularly.

That is where user research comes in. For this project, I reached out to several customers and booked three calls in which I asked variations on the same two questions with which we started our review:

• What could be improved with our documentation?
• What do you like about our documentation?

I wanted to focus each call on issues rather than getting caught up in trying to solve a specific problem on the call.

In exchange for the time participants gave to answer questions, I offered to answer any questions about our product and provide support where needed.

Through this process, I was able to guage how external stakeholders felt about our documentation. I was also able to discover specific ways in which our documentation could be improved to help them.

Scoping

With feedback from internal and external stakeholders documented, we were now ready to start defining a scope for changes. This scope would dictate how much time we wanted to spend working on different areas of the documentation.

We agreed that rewriting pages was in scope, where pages needed substantial improvements. We agreed on editing pages to bring them into a consistent style. We decided to revisit the navigation structure, especially the Deployment secction of documentation (an area for improvement highlighetd by the field engineering team).

When you scope changes based on a documentation review, you should aim to address as many of the areas for opportunity that you identify. With that said, the amount of time you have available will necessarily limit what you do.

One prominent discussion was not about the documentation itself but rather the tool we used to host our product documentation. Alternative tools offered features like more robust support for parsing OpenAPI documentation and auto-generating pages for endpoints as well as more flexible navigation structures. Ultimately, we decided not to change tool because we could get more from our existing tool with some changes, but to keep this in mind for the future.

Making the changes

Now came the fun part: updating the documentation. This was and is a heads-down process. I read over pages across our site, making edits where appropriate. I added stronger first sentences. I removed obsolete sections. I worked with teams on sections where I required more information to make edits.

Across a site with 70+ pages, I spent around a week of focused time working on changes. Most of my role was editing, with minor changes. In some cases, I wrote new pages.

One of the highlights of the experience was addressing a deficiency in how we documented our deployment solutions, which are used to run computer vision models on hardware like a Raspberry Pi or an NVIDIA Jetson. Up until the review, we were focused more on the how. We did not have as much contextual information that would be useful as a reference. We identified a few new pages to create, which included information on:

• A matrix of deployment options and what models were supported by each option, and;
• Models whose weights you could upload to our platform, among other topics.

We also used this time to revisit our API documentation. While our routes were documented, we re-ordered the documentation so that routes were roughly in order of the typical user's journey. For us, this meant we featured "workspaces" (a group of users) before "projects" (that are associated with a workspace), which were both before API endpoints on topics like model training and using a model. Adding linear progression to the navigation structure makes it easier to find the endpoint for which you are looking.

I am not going to list everything we did, but some other tasks we worked through included:

1. Adding new screenshots
2. Updating existing screenshots where our user interface had changed significantly
3. Updated the homepage to link to relevant resources
4. Deprecated documentation that was no longer relevant
5. Moved long scripts into their own GitHub repository
6. Added documentation for more Python methods
7. And more

We didn't incorporate all feedback. We prioritized feedback based on time required and expected impact for customers.

Announcing the project, and review

With the changes ready, we announced internally that an updated documentation site was coming soon that improves on our existing library of documentation. We shared the site for everyone to look at and launched it to the world. This was the exciting part: the culmination of weeks of thought, review, discussion, writing, and editing.

We have heard positive feedback from customers (which team members have kindly highlighted to me!) about how our documentation helps them use our platform. We recieved praise internally for the project, too. After working on this project, I feel more confident in our documentation being easier to navigate, up to date (at the time of editing), and consistent. This was important to me because, as a writer, I could see opportunities for improvement that I wanted to address.

Since this project, I have led two reviews for open source projects. Both reviews were on a much smaller scale. One was prompted by a dry run (which I recommend everyone try with documentation!) and the other was prompted by internal friction around how to use an open source package. In both cases, seeing someone experience friction with a product ignited us to action. As a technical writer, it is your reponsibility to be the person that takes friction and turns it into tangible results.

Through holistic documentation reviews, you can catch the little details that you might miss in your day-to-day. Through a holistic review, you will identify opportunities for improvement that make your documentation more usable, intuitive, and complete for users.