Search
Jamis Buck
Repurposing the Rails CLI
Edit
#mongodb
#command-line-interface-cli
#ruby
#activerecord
#database-integration
Summarized using AI

Repurposing the Rails CLI

Jamis Buck • September 26, 2024 • Toronto, Canada

The video, titled "Repurposing the Rails CLI," features Jamis Buck, a former Rails core contributor and Capistrano creator, discussing how MongoDB enhanced the Rails Command Line Interface (CLI) for better integration with Mongoid, their Object Document Mapper (ODM) for MongoDB. This presentation, delivered at Rails World 2024, aims to empower developers to extend the Rails CLI for their own needs, focusing on the following key aspects:

  • Context and Purpose: Jamis emphasizes the gap in Rails' support for non-SQL databases like MongoDB, highlighting the challenges users face when trying to integrate MongoDB with Rails applications. Traditional Rails applications favor SQL databases, which limits MongoDB's usability directly through the Rails CLI.

  • MongoDB and Mongoid: He introduces MongoDB as a document database alongside Mongoid, which serves as an Active Record-like interface for MongoDB. This allows users accustomed to Active Record to transition smoothly to Mongoid without much friction.

  • Frustrations with Rails' Opinions: The talk addresses the opinionated nature of Rails, which inherently disqualifies MongoDB from direct CLI support. Buck outlines the labor-intensive process users must follow to create a new Rails app with Mongoid, illustrating how it involves multiple steps like skipping Active Record and manually installing Mongoid.

  • Creating a CLI Extension: Jamis describes his guiding principles—appreciating Rails’ opinions while advocating for MongoDB users. He elaborates on his approach to extending the Rails CLI using Thor, a tool from Rails that allows for building command-line applications.

  • Implementing the Solution: He explains the process of creating a custom CLI that automates MongoDB setup for Rails apps through a template evaluated by the Rails CLI. This allows developers to create new Rails applications with MongoDB using simplified commands.

  • Monkey Patching: Jamis discusses techniques such as “monkey patching” to modify existing Rails command behavior, enabling the creation of a MongoDB-specific console command alongside the traditional Rails console.

  • Future Considerations: The talk closes with Jamis indicating ongoing work on this tool, emphasizing the importance of keeping compatibility with future Rails versions. He encourages a deeper exploration of Thor and the Rails CLI, suggesting that understanding the CLI can lead to innovative enhancements.

The primary takeaways from this presentation are:
- Developers can effectively extend the Rails CLI to accommodate MongoDB integration through thoughtful design and implementation.
- Open-source projects like MongoDB/mongoid-rails are important for the continuous improvement of these integration processes, demonstrating collaboration within the Ruby community.

Repurposing the Rails CLI
Jamis Buck • September 26, 2024 • Toronto, Canada

At MongoDB, they wanted to add a tighter integration between Rails and Mongoid (their ODM), so they created their our own CLI tool that extends the Rails CLI, adding the additional functionality they seeked. Former Rails core alumnus and Capistrano-creator Jamis Buck shows how they did it at #RailsWorld, and how you can do it yourself.

Thank you Shopify for sponsoring the editing and post-production of these videos. Check out insights from the Engineering team at: https://shopify.engineering/

Stay tuned: all 2024 Rails World videos will be subtitled in Japanese and Brazilian Portuguese soon thanks to our sponsor Happy Scribe, a transcription service built on Rails. https://www.happyscribe.com/

Rails World 2024

00:00:08.960 All right, so I had an amazing talk prepared. I was going to announce all kinds of really cool new things, and then David gave it. What a jerk! No, I'm going to talk today about how to take the Rails CLI and make it do your bidding.
00:00:16.039 Specifically, I'll discuss how we at MongoDB took the Rails CLI and used it to serve the needs of our users that the Rails CLI itself was not able to meet. I hope that by the end of this talk, you all can feel confident to try some of this yourself and see what you can do with the Rails CLI and the tools that help build it.
00:00:31.679 I also want to mention that I really had a lot of fun with this project that I will be describing here. As a Rails core alumnus, sometimes people think I know everything about Rails, but the fact is, I'm just as delighted as the rest of you to dig in and discover new things.
00:00:49.280 Since I was on Rails core, a lot of new things have been added, so it's always a journey of discovery. I hope none of you are afraid to make that journey at any time. Let's start with some vocabulary, so you know what I'm talking about when I mention certain terms. First of all, MongoDB is a database, but it is also the company that makes the database. It's the company that pays me to write Ruby code, and I am incredibly grateful for the opportunity to make a living contributing to the Ruby community.
00:01:23.560 How many of you are familiar with MongoDB, the database? Good! And how many of you have used it in anger? How many of you have used it with joy? No need to answer that question. So yes, a lot of you are familiar with it, which is fantastic. You know that it is a document database, and it is schema-less, which will become significant shortly. Next, Mongoid is one of the products we create at MongoDB.
00:02:08.959 I work on the Ruby driver for MongoDB, but I also work on Mongoid. Mongoid is our Object Document Mapper, or ODM, which is basically Active Record for MongoDB. Now, why do we have an Active Record for MongoDB? Well, it's because Active Record only works with SQL databases. If your database doesn't speak SQL, it will not work with Active Record. And tada! That's MongoDB! So we have our answer to Active Record, which is called Mongoid.
00:02:49.480 Mongoid is built on Active Model, so it can exhibit a similar interface to Active Record. In fact, we try to be as API compatible as possible to make it easier for people transitioning from Active Record to Mongoid. A lot of the syntax is similar, but naturally, there are differences. It's the gaps between what Active Record can do and what Mongoid can do that often trips us up. Now, let's talk about Rails. How many of you have heard of Rails? Just kidding, you all know Rails.
00:03:44.480 Now, this might shock you, but Rails is opinionated. I don't know if you guys knew this, but it's pretty opinionated. Usually, that is a wonderful thing, right? That’s why we love Rails: because it has opinions. We may not always agree with them, but knowing that Rails is in good hands means we can trust it to do its job very well. It's quite frustrating when it doesn't align with my opinions. One of Rails’ opinions is that it prefers relational databases that speak SQL. For instance, we have Active Record, and on top of Active Record, you've got Active Storage, Action Mailbox, Action Text—these frameworks built on top of Active Record inherit its opinions.
00:04:39.160 If your database doesn't speak SQL, it can't play in those sandboxes. MongoDB is not typically considered a relational database and is definitely not in the target audience for Active Record. Therefore, we feel some pain when trying to deal with certain features. It would be fantastic to tell our customers to start a new Rails app simply by using one line: 'use MongoDB'—bam, you're good. Unfortunately, MongoDB is not a valid option for the database parameter in the Rails CLI.
00:05:40.520 The Rails new generator specifically hardcodes a list of supported databases. It's opinionated, right? You can even have an SQL relational database with an adapter for Active Record, but if it's not in that hard-coded list, sorry, you can't use that database parameter when you start your app. This means we have to document a bunch of hoops that our users have to jump through to get started with Rails. First, they must create a new app while skipping Active Record. Technically, you can leave Active Record in, but if you're not using it, it's just extra memory and resources.
00:06:40.680 Next, they need to 'gem install mongoid,' then 'bundle install' to get Mongoid, and then generate the skeletal configuration file for Mongoid. If they're using some of our enterprise features, such as field-level encryption or queryable encryption, there are another three or four hoops they have to jump through. This is all very boilerplate, and it would be nice to wrap this up in a little bundle and hand it to our users with a command to run to make it work.
00:07:07.000 So, in trying to accomplish this, I've had two guiding principles that help prioritize and execute this correctly. First, Rails is absolutely not obligated to support everyone. One of the things we love about Rails is that it is opinionated and says no to features. If it weren't, many of us would not be very happy using it—it would feel cumbersome and clunky. So I am grateful that Rails core says no, even though I wish they wouldn’t say no to me personally.
00:08:10.760 The second principle is that MongoDB users deserve nice things too. Some of you may find yourselves in a similar position with your customers—they deserve nice things. Just because Rails says no does not mean they are second-class or unloved. Everyone is worthy of love. So, Rails is under no obligation to say yes, and MongoDB users also deserve nice features. This brings us to the question: how do we turn this situation into something workable?
00:08:47.840 We recognize that the Rails CLI is not going to do it for us, but we need to understand how the Rails CLI is implemented. Can we do the same thing? If you’re familiar with Thor, you'll know that it is an official tool in the Rails repository for building command-line applications. It is efficient and simple, providing a class called Thor, which we can inherit from; every public method becomes a command. For example, in a cloud storage application, your commands would be 'cloud:list', 'cloud:upload', etc.
00:09:29.440 A straightforward example of this in Rails would be the DB console and commands like 'rails console' or 'rails routes' that are implemented as Thor subclasses. Each method performs a specific task well without the need for complicated logic. For more complex tasks, there are Thor groups, which are combined into one monolithic command—a little bit like a recipe.
00:10:12.760 For instance, when generating a new controller with Rails, it creates files and inserts specified actions through various programmed actions. The 'rails new' command is the application generator we want to extend, though you never invoke it any other way. This raises the question: Can we extend it? The answer is yes. If I said no, this would have been a short and unfortunate talk.
00:11:03.280 In David's keynote, he mentioned that templates can be very helpful for generating the same class repeatedly. I discovered buried in the help text for 'rails new' is a line about a template—a Ruby file evaluated in the context of the application generator. This means any code you put inside that template has access to the DSL used to generate the application, which can drastically streamline the process.
00:12:00.520 For example, a hypothetical but functional template could be used to create a new Rails application with MongoDB via Mongoid. First, it would add the mongoid gem to the Gemfile, then run 'bundle install' to install it, and finally run the Mongoid config generator. These steps can now be summarized in one line, even if that line is somewhat user-hostile by requiring the explicit skipping of Active Record.
00:13:01.920 My first instinct was to wrap this up in a shell script. This naive solution checks the first argument; if it's 'new,' it adds our arguments, otherwise it just passes through everything to Rails. It worked surprisingly well! The drawback is what if someone wants to use both Active Record and MongoDB? They could install both gems, but with my shell script, we would be skipping Active Record by default, which would not support that use case. We would need a way to allow someone to point to Active Record.
00:14:25.740 What if we wanted to make the Mongoid namespace implicit? If someone wanted to create the config file, they shouldn't have to type 'mongoid:' in front of everything. This shell script could become cumbersome trying to handle that. MongoDB users also utilize encryption options, which means we need to add support for that too. Instead of complicating our shell script with all these additional features, let’s rewrite it in Ruby, which is more enjoyable to work with.
00:15:24.560 However, at this point of realization, I thought—why not just extend the Rails CLI directly? I was feeling motivated and excited, thinking it wouldn't be difficult, and thankfully it was easier than I initially feared. In our CLI file, we simply set up extensions and require the Rails CLI file, allowing Rails to handle the heavy lifting it is known for, which has been battle-tested.
00:16:34.360 The fun part is the extensions, and let's focus on one in particular: the app generator. We wanted to add some new command-line options and default settings, which involves redefining the existing parameters in the Rails CLI API. We could set defaults for the template path and the skip Active Record option. This means when someone runs our command, they'll automatically pick up my template, with the option to choose a different one if they wanted, and skipping Active Record can default to true.
00:17:22.640 We can add our own command-line options for encryption by including our new module in the app generator. This 'monkey patching' allows for these overridden options to work seamlessly. It felt like a hallelujah moment as everything started working correctly. Now, the command 'rails new --skip-active-record app_name' will give you a new app set up with Mongoid, and 'rails new --encryption app_name' will set everything up automatically and download all the necessary files.
00:18:33.480 The necessary logic must also be in your template. For instance, if you check for the encryption parameter, you can conditionally execute certain actions. Template usage isn't as widely known, making it an underappreciated feature of Rails that can provide neat solutions. Some finishing touches can make this even friendlier. For example, there’s the issue with 'rails db console'—it doesn’t work with MongoDB.
00:19:42.560 So, we want it to invoke the MongoDB console instead of the Rails DB console. One option is to monkey patch the existing DB console command, but this would lose connection to the Rails command. Alternatively, we could monkey patch the Rails CLI itself so that it looks for the Mongoid namespace first, allowing our implementation to be found first.
00:20:54.440 To implement this, we need to understand how Rails assembles its list of possible command options when invoking commands like 'rails db console'. The default list of command identifiers includes multiple entries with 'rails:' at the front. By intercepting this process, we can prepend our custom Mongoid identifiers to the list, ensuring that our implementation is found and executed first, allowing us to preserve compatibility.
00:22:19.200 To finalize, we create a new DB console command that boots the application, reads the configuration, and executes the MongoDB shell smoothly. Now when I type 'rails mongoid db console', it correctly finds and launches my version—the MongoDB console. This pattern allows me to leverage this ability for other commands as well, which I plan to do.
00:23:52.160 Another helpful enhancement would be to ensure that anywhere in a Rails project, typing 'rails something' finds the appropriate bin/rails executable. For my MongoDB commands, I wanted it to find 'bin/rails:mdb' as well. To facilitate this, I’ve adjusted the lookup to check for 'bin/rails:mdb' instead of just 'bin/rails', so I can maintain consistent behavior with my Mongoid setup.
00:25:18.080 The final template adds both the Mongoid and Rails Mongoid gems in the Gemfile. It checks for the encryption option and installs the required libraries. It emits the necessary Mongoid YAML initializer files, creates the bin/rails:mdb stub, and links bin/rails to bin/rails:mdb, ensuring users can still utilize the traditional Rails command, effortlessly integrating this new functionality with familiar commands.
00:27:14.640 You can see the current iteration of this project at MongoDB/mongoid-rails on GitHub. It's still a work in progress, yet we're getting closer to a version 1 release. We need to ensure compatibility with Rails 7.2, as I've primarily been testing against 7.1. With version 8 coming out, it’s essential to verify all functionalities work, but I’m confident in the robust tests we have in place. Overall, this project has been an incredible learning journey. Many features and different pieces of logic come together in Rails CLI, and diving into the code has offered a wealth of fun and insight. I encourage everyone to explore Thor, the Rails CLI, and appreciate the magic involved in its design. Thank you!
00:30:01.320 If you have any questions, I'd be happy to take them. How much time do we have?
Jamis Buck
@jamis
Explore all talks recorded at Rails World 2024
+31