00:00:11.389
Welcome to the testing outside the box track. I don't know where Sara is, so I'm just going to get started, and hope that's okay. You might recognize this image; it's actually one of four versions of Edvard Munch's "The Scream." It's known as one of the greatest depictions of existential angst in modern art from about a hundred years ago, and it probably describes pretty well how most of us feel about documentation because literally everyone hates it.
00:00:29.640
You are probably here because you hate documentation too, and you've heard from other developers how much they dislike it. The truth is, it's not just your average, everyday developer who feels this way. It's coming from titans of industry; it's coming from Kent Beck. He described, in an interview I read not too long ago, how projects start with ambitious ideals, promising to do it right, and then real life hits. Business hits, and it just doesn't happen.
00:01:03.359
He said we should learn from the experiences we have as we do our jobs and use those experiences to iterate. Maybe people don't maintain detailed documentation because it isn't actually a good idea. He expresses this sentiment with a delightful analogy: if it hurts to run your head into a brick wall over and over, then stop doing it. I think that's a common feeling many of us have about documentation; it just hurts too much. It hurts to create and maintain it.
00:02:01.890
And it's not just Kent Beck; it’s also all the signatories of the Agile Manifesto who explicitly stated, as one of the four guiding principles, that we want working software over comprehensive documentation. They set up a dialectical tension between the two—either you can have your working software, or you can have comprehensive documentation, but you can't have both. Well, I'm apparently very greedy; I want to have both.
00:02:29.760
I don't think it's unrealistic, despite the collective wisdom from Snowbird in 2001 when they wrote that manifesto. I think there are three reasons why I’m not crazy for wanting both. The first is that the kind of documentation that these sources are discussing isn't the same kind we're going to talk about today. Today, I'm mainly discussing API documentation, specifically JSON APIs. The ideas are applicable to many other tools, but we're going to focus on APIs.
00:03:06.160
What we're primarily discussing is usage documentation—it's what you provide to your users so they can effectively use the product you gave them. If they don’t have access to this documentation, they essentially don’t have working software because they can’t use it. The type of documentation these titans of industry advise against is implementation-focused—things like UML diagrams and big upfront design documents. They suggest we build the system once in a direct and straightforward manner instead.
00:03:42.390
However, the documentation you provide to users is critical because without it, they don’t have anything at all. You don’t have to take my word for it; a person who goes by the moniker 'agile doctor' suggested it would have been better if they had written in the Agile Manifesto "working software over comprehensive requirements and design documentation," because that's what everyone was railing against.
00:04:08.519
Just as code comments have mostly fallen by the wayside in favor of self-documenting code, the idea of upfront design has also been replaced with just-in-time design. The kind of documentation we're discussing today is not an extra burden to complete every time you launch a new feature or make a change. Instead, what if it was a vehicle that helped you move forward faster and reach a better place? That’s the kind of workflow I want to describe today. Even if you don’t agree with what I’ve presented thus far, you probably still need documentation, so I hope you’ll join me for the ride as we discuss how to have a better experience with your documentation.
00:05:04.360
To address this issue, we need to understand why it is so difficult to produce accurate documentation. I will go through a few different ideas I’ve heard over the years that people usually express to explain why documentation isn't as good as it could be for their projects, and we will evaluate them one by one. The first explanation we turn to is human error. Human error is always cited as a problem, but it’s never a solvable one—we are always going to make mistakes because we are, after all, human.
00:05:40.870
If we blame human error for our documentation issues, we are focusing on something we can’t solve. We will always be forgetful, just like sticking to an average New Year’s resolution can be difficult. Let’s find a better reason why our documentation suffers. APIs are in a constant state of change; we’re always adding or removing things, resulting in a lot of churn. This highlights a psychological aspect where updating documentation becomes mechanical and mundane, making it frustrating for many creative thinkers.
00:06:39.890
There’s also an irritating trend in the community where, instead of blaming human error, we blame human malice, suggesting that developers don’t care about their users. I don’t think that’s accurate; writing accurate and effective documentation is challenging, and blaming developers is counterproductive. I believe that complexity is at the heart of the problem. We create extremely complex systems with numerous moving pieces. Since the 1950s, the generally accepted number of things we can hold in our heads at once is 7, plus or minus 2. Try to find an endpoint in your API that has fewer moving parts than that—you probably won’t find one.
00:07:39.020
When trying to manage complex systems, mistakes are inevitable. What we must avoid is the guilt trip approach. It's tiring to hear about sources that attempt to shame developers into producing better documentation. Instead, we should stop pointing fingers at the problem and start focusing on solutions that empower us to produce better documentation. Today, I will describe a system to help us do just that, grounded in the idea that we shouldn’t keep so much information in our heads because the more we hold, the worse the situation can become.
00:08:28.640
So what if we didn't actually have to remember anything when writing our documentation? There's one effective method that allows us not to have to remember anything at that point: write your documentation first. While it may seem trivial to suggest this method, its impact is significant. This approach shifts the conversation from fixing documentation to fixing code. This is important for two reasons. First, this is how our users perceive our software—if a software product does not function as described in the documentation, it is considered broken.
00:09:56.890
Second, as software developers, we have developed tools over the years that assist us in writing accurate code and identifying regressions when they occur. The community in this space has pushed testing to the forefront, highlighting its importance. If we build a test that fails until the code aligns with the documentation, we can ultimately create accurate documentation. So, throughout the rest of this talk, we will explore the idea of documentation-driven development.
00:11:30.080
This term has become popular on the internet, but many who talk about it misinterpret it as merely documentation-first development. While documentation being prioritized is helpful, what’s more significant is the way in which documentation can drive development. I’m not going to address what happens when code is deleted as that’s relatively intuitive. Let’s focus on two processes: documenting a new endpoint and updating an existing endpoint.
00:13:11.510
For a new endpoint, the process is straightforward: first, document the endpoint. As soon as you do that, you need something to drive you all the way through to completion in your code. You write a test that asserts you no longer have a fully tested documentation suite. Once you have that failing test, you need to write a test for that endpoint, and as with any driving development approach, create just enough code to make that test pass. While this may seem like a single line, it encapsulates your entire current development process, including the approaches you use to write your code.
00:14:51.970
For a change to an endpoint, the process remains similar: first, update your documentation immediately. If changes occurred, that means a test will fail because the documentation is no longer in sync. Your development workflow consists of writing just enough code to make that test pass, which illustrates that documentation is guiding you through the development process.
00:15:53.330
In contrast, many of us typically follow a user-driven documentation update process. Here’s how it works: for a new endpoint, you implement it and then document it—hopefully. You might forget to document it entirely, and then a user will file a report stating, "your API is broken." You then acknowledge this feedback and update the documentation. This process creates a lot of frustration and pain. The ideal approach would be to transition to a mindset that fosters a healthier relationship for developers and API users.
00:16:27.560
Before I continue discussing documentation-driven development, allow me to introduce myself. Hi everyone, my name is Ariel Caplan. You can find me on Twitter and GitHub as amcaplan, which is where I publish most of my work. I operate a site called amcaplan.ninja, which contains a more extensive blog post that elaborates on the tools I'll discuss today. If you’re interested to see a wealth of details, catch some gotchas, or pick up some pro tips, take a look at my site, as it can provide your coworkers a great reference.
00:17:42.620
I work at Vitals, and I'm joined here today by many of my coworkers. We operate in the healthcare space, creating transparency around healthcare data to empower consumers to make better decisions concerning the cost and quality of their healthcare providers. If you're inspired by our discussion today or the two preceding talks by Liz on accessibility and Gretchen on our high school entrance program, I welcome you to reach out to me. Additionally, I run a project called the Dev Empathy Book Club, which aims to create actionable paths for improving soft skills within the software development arena. We read one book every two months, hold discussions, and maintain an open Slack channel—everyone's welcome.
00:18:43.180
Now, on a different note, I'd like to address some negativity often directed at Ruby, a sentiment I’ve chosen to voice here for the first time. I saw a blog post by a PHP developer disparaging Ruby entitled "Why I Don't Believe in Ruby and You Shouldn't Either." It had this particularly delightful line: 'the only thing holding Ruby together was a hipster coder community of twenty-something nerds, who are now thirty-something nerds.' Regardless, this RailsConf might be the last chance I can say, 'for the next few months, I’m still a twenty-something nerd.' Jokes aside, we have a diverse community here, perhaps more so than in tech overall, and it's truly special to be part of this.
00:19:59.480
Let’s get down to business—how do we get started with documentation-driven development? As with any instruction manual, I’m going to start with Step Two; you’ll see why the first step comes later. Step Two is to create documentation that both computers and human beings can read. Many tools can help achieve this, and while I’ll be scaffolding around a single set of tools, you are encouraged to explore alternatives. Swagger, or OpenAPI as it's now rebranded, is a specification for writing JSON specifications for APIs. All this means is you create a bunch of JSON that describes the structure of your API, enabling significant clarity for your users.
00:21:24.620
With Swagger, you can articulate the routes, parameters, responses, and even include explanations in the schema for users to make sense of the endpoints. When everything aligns correctly, it produces beautiful documentation, as displayed in the Swagger Pet Store. This is what swagger documentation looks like in practice; it clearly visualizes access through an attractive interface, showcases documentation testing, and offers ample opportunity for user engagement. I’ll provide just enough context around how Swagger thinks about your APIs to illuminate the key points.
00:22:48.720
Now, let’s examine how to post a new package, meaning you’re creating one. When posting it, you’ll want to delineate the necessary inputs, understand the expected outcomes if the request is successful, and clarify the notifications you'll receive if there are any failures. Here’s how this manifests in code—this is an example of a parameters object, defining the required attributes, such as destination ID, length, width, height, and weight. These are accompanied by explicit indications of whether each attribute is required and the types involved.
00:24:10.930
The output of the API, if successful, will return a package model, while a failure will provide an error model. It’s essential that we define what happens in both scenarios clearly. As we gather all this information, we can organize it detailing the route and the request method, ensuring clarity persists throughout the documentation. Using the Swagger Editor, you can test the implementations while observing how documentation gets generated in real-time. You can seamlessly play around with the process by filling in parameters and expectations through an interactive interface.
00:25:55.140
After understanding the inputs and outputs fully, we can condense our findings and document our route accordingly. I’ll illustrate that again by detailing our previous case—a new package post request in great detail, whereby if we successfully place a new package, we can expect specific responses on the API frontend. This is important because documentation should remain fluid and easily accessible, supplying suitable examples of what an API can accomplish while simultaneously offering a collaborative testing outlet.
00:27:40.610
Step Three—with our documentation in place—entails conducting tests. For this, I'll advocate for utilizing a tool called Apivore, cleverly named as it denotes a type of creature that eats bees, much like how Apivore interacts with APIs to decode and digest their complexities. Setting it up is simple: include the Apivore gem, run the standard installation commands, and establish a reference to the endpoint serving your documentation in JSON format. Write a test that examines if all endpoint statuses are recorded properly. Then, for each combination of endpoint status, conduct a test.
00:29:09.780
To provide clarity regarding implementation: you set up the basic boilerplate for your tests, target your documentation while attempting to parse it, and specify the requests you will test, noting required parameters and expected status codes. Here’s how this all comes together in code—execute your tests, track the responses, and refine them until they align with the intended model defined in your docs. You’ll soon realize the importance of adjusting your tests to meet the criteria you've set.
00:30:52.090
Once you establish documentation, a new endpoint can now be developed driven by the tests prescribed—creating clarity for how your API should interact, allowing for response definitions and guideposts for different actions. When you do this, you’ll refine what an update entails, ensuring proper flow and identification of changing characteristics promptly. This method illustrates a clean way to integrate changes in your API without sacrificing the integrity of the documentation or overall functionality.
00:32:17.221
Throughout this presentation, I’ll now take a moment to explore a more advanced detail of how documentation-driven development intersects with potential pitfalls. I’ll highlight challenges faced when adopting these practices and aim to offer strategies to navigate those difficulties. Embracing documentation-driven principles can unveil barriers and obstacles that hinder API optimization. Identify them early to facilitate greater awareness both internally and externally, and empower your team to succeed in formulating a culture that values and prioritizes documentation and user interaction.
00:33:17.740
Using swagger and Apivore together, you can create a production environment that embraces documentation-driven development fully and inherently incorporates testing elements to guide you through the development process. The challenges we faced brought to light significant issues buried within our code, highlighting faults, user experience concerns, and unproductive endpoints. Ultimately, embracing this methodology can magnify the positives that arise from documentation testing and create robust APIs that resonate clearly with their intended audiences without excessive complexity.
00:34:55.920
Finally, in closing, consider focusing on the conservation of energy and clarity in code design—implementing sound practices for consistency allows for ease of use for end-users, minimizing confusion and empowering confidence when data is accessed. Convey an underlying user-centric approach—tailor documentation with the user in mind because proper documentation serves your entire team, shaped as guidance at every development point. We have the power to establish a feedback loop that elevates both user experience and developer efficiency, creating a thriving ecosystem around our APIs.