Test-Driven Development
How to Make a Gem of a Gem
Edit
#rubygems
#bundler
#command-line-interface-cli
#test-driven-development
#code-quality
#open-source
#mocking
Summarized using AI

How to Make a Gem of a Gem

Justin Searls • November 08, 2022 • Denver, CO

In this presentation titled "How to Make a Gem of a Gem," Justin Searls at RubyConf 2021 shares valuable insights on creating and maintaining Ruby gems. He begins by introducing the fundamental process of creating a gem using Bundler, particularly the command bundle gem. This command simplifies several steps such as defining the gem's name, setting up testing frameworks, implementing continuous integration with GitHub Actions, and establishing licensing agreements. Searls emphasizes that anyone can become a Ruby gem maintainer by following these straightforward steps.

The talk is divided into two main sections:

  1. Creating a Ruby Gem:

    • Use bundle gem to initiate gem creation.
    • Customize the gem specifications like description, homepage, and change log.
    • Run tests using rake, implementing functionality step-by-step, such as calling an API. He demonstrates this by creating a gem called "board" that interacts with a public API, showcasing the entire development workflow from creation to release.
    • Conclude with releasing the gem to RubyGems by updating the version and using rake release.

  2. Making Gems Good:

    • Searls discusses principles to ensure high-quality gems, likening them to the four C's of diamonds: clarity, color, carrot, and cut.
    • Clarity: Emphasizes establishing mental space for focused development. He shares an experience creating "awa," a gem for parsing Japanese language files, highlighting the benefits of starting fresh to avoid complexity.
    • Color: Refers to handling emotions in decision-making, particularly to avoid rash decisions influenced by frustration. Searls gives an account of forking RuboCop and the lessons learned from that experience, leading to creating "standard," a simpler linter for Ruby.
    • Carat: Covers the importance of managing gem complexity, recounting his experience with "jasmine-rails" and how its complexity led to maintenance issues. He advises being cautious with configurability to maintain simplicity and assure functionality.
    • Cut: Encourages receiving feedback positively to improve the gem code quality. He shares his journey with "gimme" and its successor "mocktail," explaining the importance of quality in design and maintainability.

Searls concludes by urging attendees to create or contribute to gems, remembering to consider quality and maintainability. He expresses gratitude for the Ruby community, recounting his growth and experiences in the field, and urges collaboration in gem development.

How to Make a Gem of a Gem
Justin Searls • November 08, 2022 • Denver, CO

Want to know how to create and publish a Ruby gem? It only takes 15 minutes to learn, and this talk will show you how.

But when should you create a new gem? And how will the type of gem you make (e.g. an API wrapper, a testing DSL, or a CLI) impact its design, testing, and long-term maintenance?

Answering those questions is… harder. That's why the remaining 15 minutes of this talk will compress a decade of design missteps, dependency regrets, and versioning nightmares into actionable advice to help ensure that your next gem is a great one.

RubyConf 2021

00:00:12.240 Hello everyone! The title of this presentation is "How to Make a Gem of a Gem." I'm Justin Searls. You can find me on Twitter, GitHub, LinkedIn, and RubyGems. Over the years, I have written 39 Ruby gems.
00:00:18.000 Actually, I've already lied to you; I've written 40, but one of them I haven’t announced yet, so stay tuned. I've lied again because we've actually written 41 Ruby gems, and one of them we're going to create and release together live on stage right now.
00:00:30.640 Today, you will all become Ruby gem maintainers, whether you're in this room or watching online. The goal of this presentation is to answer two questions: First, how do I make a gem? Second, how do I make it good? It’s not as hard as it may seem if you've never created a Ruby gem before.
00:00:48.000 Part one: How do I make a gem? The right way to make a gem is by using Bundler. Bundler comes with a great command called `bundle gem`, which you can use by typing in the name of the gem. Here, I am creating a gem called "Board." This gem is kind of like an interactive fiction text adventure. It asks, "Do you want to generate tests?" and you can choose your test framework; I prefer using MiniTest. It will also ask if you want to set up continuous integration for your gem, and I like to use GitHub Actions because it’s fast and all your code is in one place.
00:01:13.360 Next, you can choose to license it as open source under the MIT License, which is common for most gems. Then it will ask if you want to include a code of conduct, which is a good practice, so I say yes. A change log is also important; it tells you what changes occurred in each version of your gem. You can check out Olivier's awesome site, keepachangelog.com, for reference.
00:01:31.360 Next, you'll want to add a code linter and formatter. Your options are RuboCop and Standard. The first contribution I made to Bundler was having this feature, as we maintain Standard at Test Double. Once initialized, Bundler creates a Git repository, and I like to run `git commit -m 'bundle gem board'` right away whenever I generate a new project.
00:01:54.160 Now that we created the new gem, let’s run `bundle install`. However, we run into some warnings. An error states that the gemspec is invalid due to all these TODOs in there, meaning we don’t actually have a working gem just yet. Let's jump into the gemspec to examine the items we need to address. We’ll start with the description. We don’t need that second description, and we need to fill in the homepage; it can just be the GitHub page. We also have to specify the source code URL and the change log URL. Now that we've resolved these TODOs, we can commit the fixes.
00:02:25.440 After fixing the gemspec, let’s run `bundle install` again. Everything installs cleanly, so now we can run `rake` to test and lint our gem. However, we encounter test failures because the default test desk for the 'board' gem is simplistic. We will delete the test because I just want to reach a passing build as quickly as possible.
00:02:54.880 Now that we have a gem that does nothing, we will make it call this free API called the Board API. We will write a simple test to ensure it returns an activity. The public API will be `board.now`, and we just need to assert that it returns an object with properties like "price" and "description." Let’s run `rake` again, but it will fail because there’s no `board.now` method on that module.
00:03:04.240 This isn't a test-driven development talk, so I'm not going to go back and forth showing every single step. Instead, let's dive in and write the code directly. In the generated file, it states, "Your code goes here," so we'll remove the comment and define a module method called `now`. We will use `Net::HTTP` to make a request, parse the response with `JSON.parse`, and then create a custom object called `Activity` to map the API response into a more Ruby-like object.
00:03:31.680 The `Activity` class will be defined using `Struct`, and we will enable keyword initialization for ease of use. We also need to require `net/http` and `json` from the standard library, so we don’t have to require any additional gems. Upon reviewing our entire gem implementation, we can run our tests again, and they will pass.
00:04:00.160 With the tests passing, we can prepare to ship our gem. The default version is set to 0.1.0, but I prefer to start with version 0.0.1 to set lower expectations. Next, I will run `bundle` to update the `Gemfile.lock`, which prevents unnecessary Git interactions.
00:04:17.680 Now, I'll modify the change log to document the version update, specifying that this version includes just the ``now`` method. After committing version 0.0.1, I will run `rake -t`. This command shows all the defined rake tasks, including a helpful task called `rake release`, which handles everything—it ensures everything is bundled, generates the gem, creates a Git tag, and pushes the gem to RubyGems.
00:04:40.720 When we run `rake release`, it prompts for my MFA code for RubyGems because I have 2FA set up. Once I provide the code, it will push the gem to RubyGems! Congratulations—now you are a gem author, but with that title comes the responsibility of maintaining it.
00:05:06.560 As soon as I push a new gem, I check GitHub, eagerly anticipating the inevitable first issue. It happened—my partner Todd pointed out that the API design didn’t make much sense; it would be better suited as a command line interface. He suggested implementing a message of the day to guide users on what to do.
00:05:30.160 Let’s explore how to write a command line interface. The tools provided by Bundler and Rake simplify this process, but it requires some additional setup. We're revisiting the gemspec to ensure the `spec.files` attribute accurately reflects the files tracked by Git. We also need to specify where our command-line binaries live, which is in the `exe` folder.
00:06:03.360 Currently, the `exe` directory doesn't exist, so we will create it, create a file, and make it executable. We need to add a shebang line to indicate that we will be writing Ruby code. The load path is usually handled for us by RubyGems, but since we’re inside the gem, we must manually load the `lib` directory to require our Board file.
00:06:23.680 For our command line interface, I want to pass the argument vector (`ARGV`) to the `CLI` class. While I don’t have the implementation yet, I will create it in advance. Now, I will commit this as a work in progress to confirm that `rake install` correctly packages the gem locally.
00:06:52.640 Running the board command shows an error, indicating that my script is working. The next step is to go back into `lib/board` which serves as the entry point for our gem and require the `cli` file that we will create. I will define the `CLI` class and its initializer, passing in the `ARGV` vector.
00:07:23.680 Initially, we don’t actually need to store it, as it allows for future proofing. In this implementation, we will simply call `board.now` and print the description to the command line. Now, we can run it locally without needing to install it to see if it correctly outputs the activity.
00:07:46.160 Step three is to create a new release; we want to stay on task. Let’s update the version number to 0.0.2 and run `bundle` again to update the `Gemfile.lock`. After modifying the change log by adding a new section for the Board CLI, I will make all these changes in a single commit for clarity.
00:08:05.680 Now I’ll run `rake release`, supply my MFA code, and successfully push version 0.0.2 to RubyGems. This is thrilling! If you check your computer later today, you can `gem install board` and run it to see how it functions.
00:08:40.879 You’ve seen everything—no detail has been omitted. Congratulations on making a gem! Now, let’s all go create gems. Additionally, if you’re interested in experimenting with creating a gem, I’ve opened up several issues regarding adding filters to the API or options in the command line interface. If you feel inspired, open a pull request.
00:09:05.679 I aim to create a welcoming environment for everyone; this isn't about excellence but about gaining experience and learning how to contribute to a gem. Now, let's move on to the second part of this talk: how do we make gems good.
00:09:21.920 When you buy a diamond, you may have heard of the "Four C’s" of gemstone quality: clarity, color, carat, and cut. We’ll explore how these relate to Ruby gems. First, let's discuss clarity, which is vital for those who struggle to focus amid a crowded codebase, myself included. I work on a Rails application called KameSami that aids in learning Japanese.
00:09:42.720 I initially built a small dictionary with 2,000 characters and 8,000 words, but I wanted to improve it significantly. To expand my dictionary, I discovered open-source Japanese-English dictionaries, but they were in XML format. My goal was to create a Ruby gem that extracts information from these XML files and converts it into Ruby objects for storage.
00:10:05.680 Operating within the monolithic structure of my application made it challenging; I love Rails for having everything at my fingertips, but it can also feel limiting. While working on the dictionary import, I made naive design decisions under pressure. I hastily pulled in Nokogiri, but the resulting implementation was inefficient, taking over 12 minutes to run and consuming an unreasonable amount of memory.
00:10:39.520 Realizing I was overwhelmed by the complexity of the codebase, I decided it was best to start anew in a fresh folder and create a Ruby gem called Awa, meaning English-Japanese. This approach allowed me to focus solely on improving the dictionary functionality, implementing a streaming SAX XML parser that greatly reduced memory usage.
00:11:07.920 The new gem allowed for a quick parsing process with tremendous performance improvements; it took only nine seconds to run. Now, my KameSami application can retrieve results for queries like "ruby," demonstrating the benefits of refactoring into a new gem.
00:11:16.960 Next, we’ll talk about color, which highlights the importance of creating gems with thoughtful design to avoid chaotic decision-making. I've found myself facing potential rash decisions in coding, especially while overwhelmed. I believe the best work comes from a balanced state of mind; ideally right in the medium, where I can think clearly.
00:11:46.480 As an example, I used to dislike linters and formatters like RuboCop because they created unnecessary arguments that detracted from productivity. However, over the years, I witnessed too many teams engage in endless debates regarding coding styles, which created a toxic environment.
00:12:15.680 In a moment of frustration, I decided to fork RuboCop and create a variant called RubbyCop that would eliminate configurability altogether. However, with ongoing developments in RuboCop, I quickly realized that maintaining a fork was not sustainable; I would constantly be outpaced by the original project. Eventually, I abandoned this fork.
00:12:40.960 This experience taught me the value of community and collaboration; instead of rejecting a tool because I have biases, I learned to contribute to the open-source projects that make my life easier when possible. This led to the creation of a gem called Standard that relies on RuboCop under the hood without incorporating its CLI.
00:13:08.160 Using Standard helps eliminate the pointless debates on style while fostering a community-driven approach to coding standards. If you want to express discontent with a particular style, rather than changing the configuration, you simply open an issue in Standard's repo.
00:13:38.720 Next, we will discuss carat, meaning the size and weight of our gems. This lesson stems from my experience when I started Jasmine Rails to integrate Jasmine into Rails for JavaScript testing. Jasmine provides a spec-like syntax for writing tests, which was crucial for conducting tests in Rails.
00:14:06.880 While initially, I was excited to launch this gem, I quickly discovered it involved too many responsibilities—I merged too many pull requests with little oversight. The complexity of this glue gem accumulated, leading me to realize this project could grow unmanageable. Although it had a lot of contributors, I relied solely on myself for maintenance, which became overwhelming.
00:14:44.800 Due to the burden of maintaining Jasmine Rails, I eventually announced that it was no longer maintained. Fast forward a few years, I encountered the same issue of testing JavaScript in Rails, but this time I took a more thoughtful approach with my new gem, Cypress Rails.
00:15:08.960 Cypress Rails is another glue gem that connects Cypress, a JavaScript web testing tool, to Rails. This new gem focuses on building a solid API and allows me to lean on the existing, more stable components. Instead of taking on too many responsibilities, I only focus on running the tests, which simplifies the entire process.
00:15:37.760 I’ve limited Cypress Rails to offer nothing but a better experience, noticing that other features were unnecessary, allowing me to focus on effective maintenance. Since I removed certain functionalities, I now have a clear testing strategy, ensuring that every release behaves as expected.
00:16:06.080 Finally, we will touch on the cut of our gems, which pertains to how we interpret feedback. At Test Double, we embrace the concept of test doubles, which serve as substitutes in our testing. I developed a gem called Gimme to provide an alternative for existing mocking libraries with a clearer mindset.
00:16:25.840 Gimme focused on simplifying the testing process, but during my journey, I received valuable feedback from my mentor, Jim Weirich. While attending a conference, I boldly pitched Gimme to him, and although his feedback was positive, I later learned that he aimed to improve my work.
00:16:51.760 Jim highlighted the importance of having unit tests alongside integration tests, which drove home the point that I could improve Gimme’s code quality. Ultimately, I paused development on it for ten years, but recently, I decided to create a new gem called Mocktail.
00:17:17.680 Mocktail is a fresh start; it incorporates much of what I learned during my struggles with Gimme. I've invested time in crafting the API, ensuring it adheres to modern expectations of maintainability, and I’m proud to announce that I’ve focused on producing high-quality code.
00:17:45.680 In addition to developing Mocktail, I am pleased to inform you that I’ve strived for 100% code coverage, ensuring my gem is resilient to future changes. I want it to be a model of well-structured Ruby gem development.
00:18:05.760 Over the years, even though I encourage experimentation and fun programming, I have never regretted taking the time to be thorough and intentional about creating a gem. I urge you to explore whether there is a gem you could create or contribute to.
00:18:33.680 If you embark on this journey, feel free to reach out to me for guidance, questions, or feedback. Furthermore, I want to recognize this RubyConf as a milestone. It marks the ten-year anniversary of my first Ruby conference where I spoke, which was at Rocky Mountain Ruby in 2011.
00:18:57.280 Back then, Marty Hot took a chance on me, leading to unforgettable experiences alongside industry heroes like Jim Weirich, who changed my life immensely. Following this transformative decade, I’m now proud to lead a company called Test Double that has evolved significantly.
00:19:22.000 Test Double now consists of nearly 100 consultants, many of whom are present here. Working with amazing clients like GitHub, Zendesk, and more fills me with gratitude and joy. If you share the vision of improving software development, consider learning more about consulting opportunities at Test Double.
00:19:47.120 If you find yourself undertaking ambitious projects at work or could use additional support, Test Double consultants can assist your team with legacy challenges or even enhance your new skills. I encourage you to share this with your managers or stakeholders.
00:20:16.080 Lastly, I want to take a moment to thank all of you for being part of my journey. Over these ten years, my life has transformed thanks to the Ruby community. Test Double appreciates the connections we've developed and how we've collaborated to foster lifelong friendships.
00:20:41.919 Your engagement and contributions have shaped me, and I am incredibly grateful for the time we have shared today. Thank you, everyone!
Justin Searls
@searls
Explore all talks recorded at RubyConf 2021
+95