Software Architecture
The Little Engines That Could
Edit
#rails-engines
#ruby-on-rails
#software-architecture
#rails-plugins
#microservices
#developer-experience-dx
#testing
Summarized using AI

The Little Engines That Could

Mina Slater • May 17, 2022 • Portland, OR

In the RailsConf 2022 presentation titled "The Little Engines That Could," Mina Slater introduces the concept of Rails Engines to developers of varying technical backgrounds. The presentation aims to demystify Rails Engines, exploring how they function, their benefits, and how they differ from other Ruby and Rails constructs.

Key Points Discussed:
- Introduction to Rails Engines: Slater explains that Rails Engines are not standalone but require a host Rails application to operate, similar to a symbiote in Marvel Comics, which relies on a host to thrive.
- Benefits of Using Engines: The audience learns that Rails Engines can encapsulate related code, enhancing modularity and reducing the cognitive load for developers by breaking complex applications into smaller, manageable pieces. This can also improve testing efficiency by narrowing the focus of tests.
- Comparison with Gems and Plugins: Slater clarifies how Engines relate to gems and plugins, noting that while all Engines are plugins, not all plugins are Engines. Engines offer more extensive functionalities compared to regular gems and are tightly integrated into the Rails MVC architecture.
- Types of Engines: There are two main types of Engines: full engines and mountable engines, where mountable engines can run alongside the host application and can share class names without conflicts through namespacing.
- Implementation Steps: Slater guides the audience through the steps of creating a mountable engine using the Rails plugin generator, providing an overview of the generated file structure and essential configurations needed for testing and integrating the engine into a host application.

Significant Examples:
- The metaphor of a symbiote like Venom is utilized to illustrate how Rails Engines enhance a host application by providing functionalities it lacks, much like how the symbiote grants superhuman abilities to its host.
- A hypothetical case scenario of a consulting company tasked with building an engine for restaurant reviews further solidifies the concepts discussed, showcasing practical applications of Rails Engines.

Conclusions and Takeaways:
- Slater emphasizes that while this presentation offers foundational knowledge, the topic of Rails Engines holds much deeper complexity and potential applications that warrant further exploration. She encourages developers to consider how they might incorporate Engines into their projects, noting the importance of understanding when and why to use them effectively.

Overall, Slater's presentation serves as an encouraging introduction to a resource that is underutilized yet powerful within the Ruby on Rails framework.

The Little Engines That Could
Mina Slater • May 17, 2022 • Portland, OR

Rails Engines. It’s more than just a cute name made up of two words both related to trains.

Are they plug-ins? Are they microservices? When do we use them? How do we implement them? Why aren’t they used more often?

Those are the questions I wish were answered for me when I first learned about Rails Engines. Inspired by Wired’s Explain In 5 Levels series, we will explore Rails Engines and address these quandaries using a variety of techniques, breaking down what engines are and how and when to use them.

RailsConf 2022

00:00:12.120 First, I want to thank Ruby Central and all of the Rails organizers and volunteers for having me present this week. If you want to follow along with the slides on your device, you can find them at the link at the bottom.
00:00:30.300 So, hello! I'm Mina, and I use she/her pronouns. I have been a Rails developer and consultant since 2018. I will be available on the RailsConf Slack and checking Twitter periodically, and I love making new friends, so I hope you'll reach out to connect.
00:00:41.340 Now, to make sure that you're all in the right place, this talk is called "The Little Engines That Could." We'll help people from three different levels of technical experience understand the basics of Rails engines. While brainstorming my proposal for RailsConf this year, I was really drawn to the explainer-like M5 track next door. I tried to crowdsource topics that my co-workers might want to learn more about. I received a lot of great suggestions, with my teammates expressing interest in subjects like GraphQL, SQL versus Active Records, and, of course, Rails engines.
00:01:20.040 Just when I thought that the Ruby and Rails community couldn't get any more delightful, I learned that Rails has something called engines. More than the satisfaction of my coding-loving heart, I was astounded that I had never heard of this term before. Up to that point, I had been working in Rails for almost four years, yet it was the first time that this term had come up. So, I decided I was going to learn something new.
00:01:52.900 The philosophy I stand by and try to pass along to aspiring speakers is that no matter how basic we think our topic is, there will always be an audience for it. Wherever we are in our careers, by sharing the things we learn, we are essentially addressing ourselves in the past and those who are where we were. This talk came about in pretty much exactly this way: in my excitement to learn a little more about the framework that I love, I went digging for information and hands-on experience.
00:02:30.840 For a community like ours that has pretty comprehensive resources, engines don't get used or talked about as much. I’m not here to make anyone an expert— we need more than half an hour for that. My goals were to use this presentation as a catalyst to dig into something new and share a summary of my learnings with all of you. My format today is inspired by a Wired YouTube video series called "Explain in Five Levels of Difficulty," where experts talk about a high-level subject in five layers of complexity: first for a child, then a teenager, an undergrad, a grad student, and finally a colleague.
00:03:05.700 Today, I'm going to introduce Rails engines to people from three different technical levels. Level one consists of the non-technical members of our software team or those new to coding, including our designers, product and project managers, and sales and marketing associates. Level two includes early-career developers, or new developers looking for their first professional roles. They know the very basics and may have built a couple of real side projects.
00:03:29.280 Finally, level three is where I would categorize myself: we are the developers who have been using Rails professionally for a few years but have never worked with engines before. I'll be diving a little deeper with each level as we advance. I had a lot of questions when I first started exploring Rails engines, and my hope is that by the end of this presentation, you will have a clearer understanding based on whichever level you identify with.
00:05:12.060 So, at level one, we're going to address what an engine is in a non-technical way. We'll explore some of its benefits and why we might choose to use one. At level two, we will look at what makes an engine unique by comparing it to familiar concepts, like gems and plugins, while also touching on a couple of different types of engines. At level three, we will briefly cover how to get started implementing an engine and discuss some things to look out for when testing.
00:05:57.539 Now, imagine that we work for a consulting company. Collectively, this company has a sales team that is pitching potential clients. The sales team members don't come from a technical background, but they work closely enough with developers to understand how to ask the right questions. One day, they inform us that we have been asked to build an engine called Critic to handle restaurant reviews for a potential client.
00:06:20.580 Before the sales team can proceed, they ask us to explain what an engine is so they can understand what this client is asking for. One of my favorite exercises is to come up with real-life metaphors for technical concepts so that I can explain these concepts in a way that everyone understands. This is a strategy I still use when I encounter new concepts. Now, to explain Rails engines to our sales team, allow me to introduce you to the Symbiote.
00:07:11.880 The Symbiote is a fictional species from Marvel Comics. It cannot exist independently; it requires a host organism to be powerful and viable. The most well-known of the Symbiotes is Venom, and it turns out this Marvel anti-hero has a lot in common with Rails engines. First, Rails engines do not operate on their own; they require integration or to be mounted into a host Rails application to function effectively. Similarly, the Symbiote cannot self-sustain; it needs a living organism to host it in order to survive and function properly.
00:07:53.120 Neither Rails engines nor the Symbiote can reach their full potential without their respective hosts. Once incorporated into an application, a Rails engine provides that application with additional functionalities it didn’t have before. For instance, the Devise gem provides our application with the ability to authenticate when it doesn’t implement that feature on its own.
00:08:14.220 Likewise, when the Symbiote bonds with another organism, the host is enhanced with powers like superhuman strength, speed, agility, and endurance. One very famous bond is between the Symbiote and Eddie Brock, which turns the down-and-out reporter into Venom, one of the most powerful characters in the Spider-Man universe.
00:08:46.920 Both Rails engines and the Symbiote enhance their hosts by providing them with the ability to do something they couldn't do before. A single Rails engine codebase can be used by multiple host applications. Each application incorporates the engine independently and can override parts of it without affecting other applications that are also using the same engine.
00:09:01.380 Similarly, the Symbiote can form bonds with more than just Eddie Brock. It also takes on the overall shape of the host it embodies. For example, Venom is shaped like a humanoid character, but when the Symbiote bonds with characters from the Guardians of the Galaxy, they look drastically different. Both Rails engines and the Symbiote are adaptable and can bond with an array of different hosts.
00:09:54.120 Now that our sales team has a better understanding of the request, they want to know what this potential client might be trying to accomplish with an engine-based design. While Rails engines are reusable, this client might want to package this engine to share its functionalities between multiple applications or open-source the code for others to use.
00:10:24.900 Because a single engine can be incorporated into multiple applications, they are a good option for sharing reusable code. Rails engines encapsulate related portions of code. This client’s Rails application might be getting too big, and their developers have difficulty holding enough context to work effectively.
00:10:55.740 Rails engines help reduce cognitive load by splitting the system into their own scopes. This way, developers only need to understand a small part of the overall system while working on the code. They also reduce the scope of tests. This client’s test suite might be getting too slow and unruly to maintain, and they want to decrease the spec execution time. Encapsulating related functionalities in an engine also reduces the runtime and organization of the test suite.
00:11:32.520 By design, an engine shouldn’t rely on any of the behaviors in the host application, so its specs should be able to run in isolation. After this conversation with the sales team, they learned what a Rails engine is and what values it might provide for our potential client. They now have a high-level understanding needed to further this conversation.
00:12:05.700 Now, imagine that our sales team has gone ahead and secured the contract to build this engine for the client, and we have assigned developers to this project. Our team includes an apprentice we are mentoring. This apprentice knows the basics of Rails and has built a couple of side projects with it, but they see Rails engines as similar to gems or plugins. However, they have heard engines described as miniature applications and are deeply confused, prompting them to ask for clarification.
00:12:35.580 In the non-technical explanation earlier, we made some comparisons between engines and applications. The Rails guides state that applications are basically supercharged engines because the Rails.application class inherits many of its behaviors from Rails.engine. At first glance, they are similar in directory structures as well, with views, controllers, models, and assets.
00:13:10.680 The biggest conceptual difference between them is that applications can operate independently, while engines cannot. An engine’s dependencies are listed in its gemspec file rather than a gem file, and when we add the engine to the host application’s gem file, we would bundle as usual. This tells the bundler to parse the engine’s gemspec, installing its dependencies alongside those of the host application.
00:13:49.900 The ability to reuse an engine's code across multiple applications often leads to comparisons with gems. While gems are packaged Ruby libraries, Rails engines can be distributed similarly; however, they will only work within a Rails application, whereas non-engine gems can be incorporated into any Ruby project, whether it uses Rails or not. The main difference is that gems are general Ruby code, while engines contain Rails-specific entities like models, controllers, views, migrations, and routes.
00:14:24.780 Plugins are also reusable code, and while engines are a subset of special gems, they are also a subset of plugins. Engines and plugins are intended to be used alongside a Rails application to provide additional functionality, but engines provide more framework than plugins. According to the Rails documentation, an engine can be a plug-in, and a plug-in can be an engine.
00:14:59.580 If we consider the geometry rule that all squares are rectangles but not all rectangles are squares, we can restate: all engines are plugins, but not all plugins are engines. The term "plugin" describes any isolated unit of code that extends or modifies the core of a Rails application, while engines are specific implementations, fully structured within Rails' MVC architecture, whereas plugins are generally meant for model-level code.
00:15:44.760 Rails considers engines to be full plugins due to their bootstrap using the Rails plugin generator. To create an engine rather than a generic plugin, we use either the --full or --mountable flag. With the --full flag, the generator creates what is called a full engine, integrating with the host application as if the engine's code was written alongside the application’s original code.
00:16:16.260 Mountable engines work more like mini Rails applications, allowing us to mount them into the host at a designated route. These types of engines run in parallel to the host application rather than integrating with it. A mountable engine includes all features of a full engine plus additional namespace isolation, allowing the host application and the engine to have classes with the same names without collisions.
00:16:58.800 Mountable engines also come with asset manifest files, layout view templates, and namespace isolation for routes and configuration classes. For our apprentice staff on the team, we were able to clarify some confusion by explaining these subtle differences and similarities between engines, applications, gems, and plugins.
00:17:34.320 Now that our sales team has secured the project and our apprentice understands what we are attempting to accomplish, it's time to build the engine. In the well-documented world of Ruby on Rails, engines are surprisingly seldom discussed. Even though our team includes experienced Rails developers, none of them have implemented an engine before, so they have asked us for guidance.
00:18:00.540 To get started, we're going to use the plugin generator to bootstrap a mountable engine called Critic. As the generator operates, we see typical outputs just as we do when starting an application. Among standard parts like controllers and models, some additional files are generated, illustrating the ways engines differ from applications.
00:18:47.160 Inside our lib directory, we have critics.rb, which serves as the entry point of our engine. This file will be required when the host application loads its dependencies and can be configured with the engine's external dependencies. Additionally, inside the lib directory, there is a subdirectory named after our engine, containing an engine.rb file. This file informs the host application that this dependency is an engine, serving a similar function as an application's config file.
00:19:30.060 Since Critic has been generated as a mountable engine, one parameter is already configured for us. This line declares that the entire engine is to be isolated in its namespace, matching our engine’s name, Critic. The Critic gemspec contains metadata about our engine and the gems on which it depends.
00:20:12.960 Once we include the engine in the host application's gem file, running bundle will install these dependencies alongside those of the host application. The generator also creates a basic Rails app inside the test directory for us, which will be used to test the engine.
00:21:01.200 We have discussed how entities in a mountable engine are namespaced inside a module, made possible by the isolating configuration we saw earlier. When we use the model generator to bootstrap a model inside our engine, it will create our review.rb model file inside a subdirectory called critic under models and automatically nest our new class inside the engine’s module.
00:21:54.000 When testing a Rails engine, a few things stand out that are worth paying attention to. Since an engine cannot operate without an application, a test directory specifically for testing our engine is created, automatically loading our engine into this testing app. Our tests will be run from its environment, which handles both manual and automated tests.
00:22:35.220 If necessary, we can also generate controllers, models, or views within the testing app to facilitate testing our engine. When making requests to a controller in controller tests, we need to explicitly tell our test app how to route those requests.
00:23:07.560 For instance, if we have a test file for a reviews controller in Critic, we might set up the routes instance variable in the test setup. This instructs the testing app environment to route requests within this test file to the engine. This ensures that a get request in the index action test will use the engine's routes rather than the host application's.
00:23:47.520 With this setup, the engine's URL helpers will also work as expected. Now, imagine we have implemented all the functionalities of the engine and are ready to mount it into our host application. To do so, we need to include the engine in the host's gem file and designate its relative path if we are developing the engine’s code alongside the application.
00:24:21.960 Next, we’ll designate a route where we would like to mount this engine. This line will mount the engine at /reviews in the application, making it accessible at localhost:3000/reviews when developing locally.
00:24:46.020 Regardless of which level you fall into, I hope you have taken away something valuable from this presentation. We have covered what engines are, how they differ from other Ruby and Rails features, and the practical aspects of implementing one. Of course, there are still many more questions I want to address, and we have barely scratched the surface.
00:25:11.820 Personally, I think the next step will involve digging deeper into what engines are capable of, when it might be a good time to use one, and what the trade-offs might be. Maybe there will be a part two!
00:25:33.540 However, as with learning anything new, we have to start with the basics. What I have provided today is a summary of the things I found most interesting and essential for getting started. I merely introduced another tool to your Rails toolbox, and it’s now up to you to decide if it is useful for your task at hand.
00:25:53.780 Thank you.
00:26:06.040 Thank you all so much for choosing this talk. I would like to plug a couple of my friends who are also speaking at this conference. Audrey Slater is talking about TDD with a pirate theme, so be sure to check that out on Thursday. Mercedes Bernard's talk on how to onboard efficiently into a new codebase also promises to be excellent, so be sure to catch that on Thursday too.
Mina Slater
@minaslater
Explore all talks recorded at RailsConf 2022
+64