Ruby Video
Friendly.rb 2023

Let there be docs!

Let there be docs!

by Svyatoslav Kryukov

In the presentation titled "Let there be docs!" at the Friendly.rb 2023 conference, Svyatoslav Kryukov humorously discusses the often-overlooked topic of API documentation. He starts by acknowledging the challenges developers face with documentation, emphasizing its perception as a tedious task. He outlines the historical context of API development, mentioning the evolution from REST to various protocols like GraphQL, while noting the ongoing relevance of REST due to its flexibility.

Key points discussed include:

  • Issues with Documentation: Kryukov reviews common frustrations, such as outdated documents leading to endless API questions, cumbersome manual testing setups, and frequent API updates causing frontend breakage.
  • The Role of AI: He playfully questions the potential of AI to eliminate documentation efforts, providing a fictitious example that humorously illustrates the pitfalls of relying solely on automated tools.
  • Documentation Approaches: The talk categorizes four key documentation strategies:
    • Manual Documentation: Straightforward but often outdated and poorly synced with current implementations.
    • DSL-Based Documentation: Ties documentation creation to code implementation, though it often leads to cluttered code and lacks test capabilities.
    • Test-First Approach: Documentation generated from tests ensures accuracy, though it is still a retrospective process.
    • Documentation-First Approach: This proactive approach emphasizes creating documentation before implementation or testing, enabling better team collaboration and earlier feedback. Kryukov argues this is the most effective method.
  • OpenAPI Specification: Demonstrating the documentation-first concept, he introduces OpenAPI as a standardized format to create API specifications, and utilizes a fun example involving animal feeds to illustrate the specification process.
  • Tools and Collaboration: He discusses tools like Swagger Editor, Spectral, and Schuma that facilitate the development and validation of OpenAPI documents, stressing the importance of collaboration across teams, including feedback loops.
  • Iterative Workflow: The benefits of working through iterations, gathering feedback early, and adapting documentation to clarify understanding are emphasized.

Kryukov concludes by underscoring the importance of adopting a documentation-first approach to foster collaboration and clarity among development teams. He encourages attendees to experiment with these practices and tools, hoping they return to their workplaces inspired to elevate documentation efforts. With the memorable call to action, "Let there be docs!", he advocates for a shift in mindset towards valuing and prioritizing documentation in software development.

00:00:05.560 Thank you! I am really happy to be here. First of all, a big shout-out to the whole team behind this friendly and welcoming event. However, the introduction from Adrian and also this tweet bring quite a lot of pressure to be the only one with a sense of humor. Wow!
00:00:12.320 But have you seen the name of my talk? It’s about documentation, which might be the most boring thing in the world, at least in my opinion. So, prepare yourself for what might feel like a 40-year journey. I warn you! OK, let’s do the talk.
00:00:38.399 In the beginning, God created the heavens and the Earth. And REST didn't create the rest. I've condensed the timeline a bit, but that's essentially where our story begins. As web applications became more complex, we asked for more advanced ways to structure APIs. We designed many protocols like JSON API, GraphQL, and JRPC with strict rules and schemas.
00:01:05.400 However, the good old REST is still going strong, perhaps because the inner artist in us resists standardization. And there is no greater nemesis for our inner artist than documentation. Oh, there are so many problems with it! Here are a few examples to see if they resonate with you.
00:01:33.000 Do you ever feel afraid to open Slack due to endless API questions from outdated documentation? Are you tired from setting up Postman collections to manually test your API? Have you ever had to roll back an API update because it unexpectedly broke the front end? I promise we will find a solution in this talk.
00:02:08.080 And since it's 2023, can we find a solution in AI? There must be a way for us to avoid writing documentation. Let’s borrow Jason's talk idea and ask ChatGPT how to bring dinosaurs back to life using the GitHub API in Ruby.
00:02:15.000 That sounds like a crazy question, right? There is no way.
00:02:29.200 Just follow these four steps and suddenly we’re in Jurassic Park! Look, there’s even a Ruby snippet over there. And you know what’s even worse? Users might actually try that and come to you with complaints. Yes, I made that example up, but the whole story is real. You know that it's real.
00:02:37.959 So, I think there is no way for us to avoid writing documentation. Now that we're all on the same page, let’s discuss four key approaches to documenting APIs. We will spend the next few minutes discussing the pros and cons of each, starting with manually written documentation.
00:03:11.000 Manual documentation is straightforward; it just involves documenting our implementation after it’s already completed. However, this method is not only boring, but the resulting documentation is also constantly outdated and out of sync with the implementation. We could potentially fix that if we could somehow bind our documentation to our implementation.
00:03:36.879 We can do that with some DSLS. With this approach, we generate documentation out of our controllers using some DSLs, and now our documentation is always up to date, which is great. But there are still some downsides, starting with the fact that documentation is still ready only after the code is completed. Our controllers become cluttered with some magic DSLs containing metadata, making it not really readable.
00:04:07.159 Finally, there are no tests to assert that our generated documentation is valid. An obvious solution is to shift those DSLs to our tests using a test-first approach. Now we're writing our tests first and adding some magic DSL into that to generate documentation. This way, we can ensure that as long as our tests are accurate, our documentation will be too.
00:04:31.039 Despite the fact that documentation is ready before implementation, it still is only ready after the tests are written, which is a primary issue with both test-first and code-first approaches. Developers might simply overlook the documentation since it's considered an afterthought. I once worked at a company with many microservices, and those approaches were used, but sometimes, the generated documentation was unclear and contained function names instead of descriptions, omitted attributes, and other oversights. It seems no one cared, but we should care.
00:05:02.759 With a documentation-first approach, our focus shifts to creating comprehensive documentation before we start coding. I call it the documentation-first approach, but you can also call it schema-first or specification-first, as we're now writing not just a mere documentation but a comprehensive specification.
00:05:40.760 This fact opens a lot of possibilities for us. Not only can we generate documentation, but we can also test our applications against it, validate incoming requests from users, and so on. It allows us to enhance our workflow. We can draft a specification, discuss it with the team, and ensure that the API meets all necessary requirements.
00:06:07.880 Next, we can add tests to confirm that specification, and finally, we implement the API. The last two steps can be swapped depending on your preference for the test-driven development approach.
00:06:39.840 With the documentation-first approach, there are no magic DSLs, so our code and tests are clean and readable. Documentation is ready before implementation or tests, so we can receive feedback earlier from our team.
00:07:01.680 Ultimately, documentation serves as a specification. The only downside is that it might require a shift in workflow. In summary, considering problems like outdated documentation, lack of automated tests, and communication issues, the documentation-first approach stands out as the most effective and collaborative approach.
00:07:15.240 Now that we have discussed the theoretical approach, let’s see this approach in action. The first step is to choose a specification format, and there are plenty of options out there, but OpenAPI is the most popular one and arguably the de facto standard. We are going to use it today.
00:07:46.880 To learn more about OpenAPI, consider reading the Petor API. It’s a great example of a well-documented API using OpenAPI. While commonly used for demos and examples, it might be a bit complex for our needs today. Instead, we’re going to assume we already implemented some features, like a simple CRUD for animals, and we received positive responses from our users.
00:08:30.239 Our happy stakeholder, the CEO of the Ark, comes to us with a new feature request to implement a feed feature. The request is somewhat vague, but unfortunately, the CEO of the Ark is swamped with other tasks and can't address our questions immediately, so we should come up with something on our own.
00:09:04.000 I suspect the CEO wants us to implement a feed similar to Twitter's, you know, to make animals destroy each other online and not in real life. That's a reasonable assumption since social networks are popping up like mushrooms after the rain these days. Let’s describe that with OpenAPI.
00:09:43.200 This is a minimal OpenAPI document. Unfortunately, explaining every keyword and feature of OpenAPI would take forever and isn't the point of my talk, so instead, I will just demonstrate a couple more slides to show how it looks and feels.
00:10:05.280 We will add all necessary operations for our feed feature, and I think it’s a great idea to document in this manner—moving from top to bottom and adding more details with each next step.
00:10:35.280 Let’s do that with one of the requests here. We've added a response description for a successful response, and note that the schema keyword inside OpenAPI uses another standard, JSON Schema, which can be used to validate JSON even outside of OpenAPI. It's a great tool that's really readable and easy to understand.
00:10:43.600 However, it can be a bit verbose. To fix that, we can use references. Here, we are referencing a local component in this document, but it's also possible to reference external files or OpenAPI descriptions to avoid dealing with long lines of code.
00:11:05.920 Next, we can iterate on this specification, adding more keywords and features until we’re satisfied with the result. Yes, maybe we got a little overboard there, but I think it’s fine.
00:11:32.880 Since we are done with our specification, we can now leverage the OpenAPI ecosystem, starting with editors. Editing a specification in PowerPoint is fun, but there are other options to consider, like Swagger Editor, which is a web-based application that can be embedded in your application's development environment, or this VS Code extension from Redly.
00:12:24.000 Next, linters. I bet almost everyone in this room uses Rubocop daily, but do you lint your OpenAPI documents? If you don’t, you should! Spectral is the way to go! So even if you use code-first or test-first approaches, adding Spectral to your workflow and your CI/CD pipeline will elevate your documentation.
00:12:48.160 Next, our front-end team can start implementation right away using mock servers like Prism, and the whole team can benefit from beautiful documentation UI generators like Swagger UI and Redoc. There are many more OpenAPI tools in this list, but the most important tool is collaboration—collaboration with our whole team, with front ends, analytics, and stakeholders.
00:13:14.000 With documentation, it’s now easy to discuss and improve features. By the way, now that we've edited, linted, and rendered our documentation, let's show it to the CEO of the Ark. Awesome work! But I wanted a feed of animals, not the animals' feed!
00:13:46.560 It seems we've misunderstood the requirements. This kind of misunderstanding happens all the time in software development, and isn't it great that we used the documentation-first approach? We didn’t implement anything yet and got feedback earlier. Now we can just reiterate on the documentation and make it work.
00:14:17.680 Assuming we did it and our documentation is ready and approved, let’s take a look at our workflow again. We just finished with the two first steps. Next, we are going to add tests to confirm the specification and implement the API. Let’s look at gems that can help us with that.
00:14:37.760 Starting with commit, I’ve worked with that gem for a couple of years—it’s a great tool, but it lacks some features from OpenAPI and JSON Schema. Next, OpenAPI First and JSON Schemer. The JSON Schemer gem just got updated to fully support JSON Schema's latest version, and I bet OpenAPI First will be updated soon to fully support the latest OpenAPI 3.1.
00:15:05.760 Finally, Schuma, which is our choice for today. There are many advantages to using JSON Schemer and Schuma. First of all, it has full support for OpenAPI 3.1 and JSON Schema. It's flexible and extensible, comes with ARPEC helpers out of the box, and has a cool name with a reference to Elder Scrolls games. And I’m the author, so obviously, it’s a 100% unbiased opinion!
00:15:58.320 Now that you know the best gem on the blog, let me show you how to configure Schuma. We only need to specify the documentation path and include Schuma into our request specs. Now we can access our OpenAPI document through Schuma.
00:16:38.399 Try saying that 10 times faster! For example, we can validate against the specification. Here's an example with a simple test. We can ensure that our OpenAPI document is always valid.
00:17:02.799 But the most important part is that we can validate our endpoints with helpers that validate request parameters, headers, and body, conforming response schema that validates response status, headers, and body.
00:17:39.839 Finally, conform schema validates both request and response. We have a vanilla request spec, and there are no magic DSLs—just a couple of helpers that validate all those attributes. Just imagine doing that with plain Ruby!
00:18:15.360 That’s insane! With this approach, we can now TDD our way to the perfect implementation of our specification. That's basically it—we found the path to solid ground—the land without proper documentation.
00:19:07.920 I've been working hard for the past couple of months on the Schuma gem and just released the first version today. It already has lots of features, but there are many more to come. If you’re interested, please try it out and share your feedback with me.
00:19:42.560 Also, consider following the project on GitHub and get your portion of Schuma today.
00:19:57.919 Now to the last part of our talk, which covers tips and tricks. It's okay to begin with the documentation-first approach on only one endpoint. Once you and your team are comfortable with this new approach, you can expand it to the entire API.
00:20:13.680 For those transitioning from code-first or test-first approaches, generating OpenAPI documents from your existing code can serve as a solid starting point. I've been in both situations, and it's worked great for me.
00:20:40.960 Next, add Spectral to your workflow and CI/CD pipeline. Once you’ve done that, use custom linters to enforce your API design, like specifying your response or pagination format, and ensure consistency across your API.
00:20:59.760 There are also real-world examples of rule sets available in Spectral documentation, which you can use as references to learn more.
00:21:23.960 Finally, please read the JSON Schema keyword PC; that will definitely level up your validations. For example, there's an annotated properties keyword that you can use to avoid exposing internal attributes to your users, like password hashes.
00:21:43.840 Just add that keyword to your schemas to ensure that your API is safe. You can even use custom linter rules to enforce this.
00:22:13.440 Thank you for your attention! If you have any questions afterward, feel free. I left some great resources for you on OpenAPI and JSON Schema, as well as these great talks I really enjoyed.
00:22:45.120 To recap: consider using the documentation-first approach to enable collaboration and early feedback. Use OpenAPI to write specifications in a language-agnostic format, so you and your team have one common language.
00:23:05.560 Also, improve your documentation with linters and tools from the OpenAPI ecosystem. Finally, use Schuma to test your application against those OpenAPI documents to save time and effort, making your code and tests more readable and your API more stable.
00:23:32.080 I hope you like this approach. So when you return to your colleagues, you can say, 'Let there be docs!' and there will be docs, and you'll see that the docs are good and separated the light from the darkness.
Edit