Checking Under The Hood: A Guide to Rails Engines

Talks

Mike Perham

2 talks

Checking Under The Hood: A Guide to Rails Engines

by Mike Perham

In this presentation titled "Checking Under The Hood: A Guide to Rails Engines," Mike Perham discusses the evolution of Rails and the introduction of Rails Engines in version 2.3. He begins with a brief introduction of himself as a Ruby developer and outlines the historical context of Rails, tracing back to its initial release in 2004 where applications were simple Ruby-based frameworks.

Key points discussed include:

- Evolution of Rails: The introduction of plugins in Rails 2.0 in 2006, allowing code sharing among developers, which addressed limitations on sharing functionality across Rails applications.

- Limitations of Plugins: While plugins enable code sharing, they cannot handle MVC components like views and controllers as they don't integrate with the Rails infrastructure.

- Introduction of Rails Engines: With Rails 2.3, engines emerged as advanced plugins that encapsulate the MVC stack into distributable gems, effectively functioning as mini Rails applications within a main application.

- Structure of Engines: Engines have their own app directory, similar to a Rails app, where wiring up routes, controllers, and views behaves like a traditional Rails app. Rails manages paths and routing configurations for engines, but developers must be cautious of name collisions between engine classes and the main application.

- Handling Assets and Migrations: Engines face challenges in handling static assets and database migrations; for example, assets must be managed manually since engines are treated as separate gems.

- Real-World Example: Perham provides an example of his own engine, Queso, which allows for dynamic database searches by wrapping a model in search functionalities. This serves to illustrate how engines can leverage existing Rails features while maintaining modular functionality.

- Future of Engine Development: The presentation wraps up with discussions on the ongoing landscape of user authentication, referencing the clearance engine as a solution to reduce redundant authentication code.

In conclusion, Perham emphasizes that Rails Engines represent a significant advancement in Rails development, enabling the sharing of complete MVC functionality alongside Ruby code. He encourages the interest of developers in exploring the potential of engines within their Rails applications.

Overall, the presentation provides an insightful view into the capabilities and challenges of using Rails Engines, reinforcing their relevance in modern Ruby development and the Rails ecosystem.

00:00:20.960 My name is Mike Perham, and I'm here today to talk to you about a new feature in Rails 2.3, which is Rails Engines. I am a local Austin Ruby developer, and I've been doing Ruby professionally for about two years now. I discovered Rails a few years ago, probably like many of you, after spending several years doing Java.

00:00:24.840 Like I mentioned, two years ago I decided I had to pursue Ruby, and I am currently working at a company called One Spot here in town. Before that, I worked at a company called Five Runs. I am also the author of the Data Fabric gem, which adds database sharding to ActiveRecord, and I'm the maintainer of the Memcache client gem, which ships with Rails and is used to communicate with the Memcache server.

00:00:35.040 So, let's take a step back in time a little bit and look back at a few years. Who here remembers the year 2004? I assume most of us were here. Ruby developers have a reputation for being relatively young. In 2004, "The Lord of the Rings" was sweeping the Oscars, and the Indonesian tsunami struck. These were two significant events in that year, but the big event for us was, of course, Rails.

00:00:40.800 DHH and the 37 Signals crew released Rails about five years ago. At that time, when you built a Rails application, it was just a Rails app—it was simply a directory containing some Ruby code. This Ruby code initialized the Rails gems, set up the Rails subsystem, and conformed to some MVC conventions. It was light years ahead of most other systems that people were using at the time. However, as developers wrote their first apps and became excited, they soon realized there was a problem.

00:00:56.520 Fast-forward two years to 2006. Who remembers 2006? Well, I don't know about you, but back then you might have been enjoying cigars in your country club because your stock portfolio was surging, and your home was worth an all-time high. Unfortunately, LOLcats had a death grip on websites at the time.

00:01:05.040 In 2006, Rails introduced the concept of plugins, which addressed the earlier problem of developers needing to copy code between applications. With plugins, we gained a way to share Ruby code and functionality easily. Rails introduced the plug-in script, which facilitated the discovery of new plugins, listing available plugins, and installing plugins. This effectively resembled RubyGems, as Rails essentially reinvented RubyGems for plugins.

00:01:18.840 Now, Rails plugins are gems; however, they are special gems that include a Rails init file, which Rails will run to bootstrap the code in that plugin. You simply add the gem to your list of gems in your configuration, and Rails will automatically add the lib directory for that plugin to its autoload path, enabling the code to be autoloaded.

00:01:23.280 While that sounds great, developers soon began noticing limitations. Plugins are excellent for adding Ruby functionality, creating classes, modifying existing classes, and monkey patching. However, they can't handle anything related to the MVC stack—they cannot add views, controllers, or routes. This limitation arises because Rails has infrastructure separate from Ruby's load path, including concepts like controller paths, view paths, and routing configurations, which plugins are unaware of.

00:01:51.600 So, let's fast forward to 2009. We elected a president, held a fantastic Ruby conference in Austin, Texas, and released Rails 2.3. The essence of Rails Engines is that they are plugins equipped with the necessary hooks to encapsulate MVC functionality into a distributable gem. Essentially, an engine is a Rails application that runs within your Rails application.

00:02:03.600 To delve into the specifics, engines, like a regular Rails application, have an app directory. This app directory contains the standard directories you'd expect—views, controllers, models, and helpers. Rails will automatically add those paths to the internal subsystem paths so it can locate your code. Keep in mind that when there are two directories competing for code, Rails will always prioritize using the application's code over the engine's.

00:02:12.240 Going deeper into models, Rails will discover any models that the engine provides in the app/models directory. However, there is a limitation to be aware of: migrations do not work with engines. You cannot have a db/migrate directory and automatically build a database for that engine.

00:02:29.100 Additionally, you should be wary of name collisions when developing Ruby code and plugins. It might not be wise for your engine to create a user class, as that’s a commonly used name. Instead, it is advisable to use modules to namespace any classes you need. Most engines in use today are adopting this approach of using namespaces or modules to prevent name collisions.

00:02:44.820 In terms of how Rails sets up the views and controllers, Rails uses code to determine if any engines are present and then adds the routing configuration, controllers, and views accordingly. It collects the routing files and the paths for both controllers and views. It's important to note that, in addition to adding view paths to Action View, Rails also adds them to Action Mailer. Engines can send email, so if you're using Action Mailer, you can include your view templates in the app/views as usual, and the engine will be able to find and use those views.

00:03:09.600 Regarding the controllers, Rails will search the engine's app/controllers directory and will load the routes from the config/routes as expected. However, a small caveat is that if you utilize helpers to load all helpers into all of your controllers, Rails will not load any engine helpers. So, you'll need to manually load that helper in any controller that requires it.

00:03:30.600 View templates work as you might expect. However, there is a limitation concerning the Rails application, which has a public directory for static assets. Since engines run as gems elsewhere, web servers like Nginx and Apache see only the public directory and do not recognize your engine. Therefore, any static assets you want to include in a public directory cannot simply reside in your engine's public directory; you will need to copy those files to the application itself so they can be served by the web server.

00:03:50.220 Engines and plugins can also add Rake tasks to the application by including .rake files in the lib/tasks directory. Regarding migrations and static assets, you can copy those files to the application's directories, which will work fine. There are two ways to address this: you can write code in Rails' init process to perform file copies upon engine initialization, or you can create a Rake task to handle it, but this requires that the user explicitly invoke the Rake task.

00:04:06.840 Moreover, plugins previously had an install.rb hook that activated when the plugin was installed; however, with the migration to gems, this hook is now unavailable. Here is some code I wrote to solve this challenge in my own engine. It's relatively straightforward, as it focuses on copying files from the engine's directory to the corresponding Rails application's directory.

00:04:25.560 One limitation that persists because we copy files manually is change management. If the Rails application is in Git or Subversion, users would need to add the copied files to their source repository and manage these files accordingly, especially when upgrading or downgrading the engine to match the corresponding versions.

00:04:42.660 Thoughtbot has an engine called Clearance, which I will mention in a moment. They go to great lengths to deal with this issue, and their code can indeed become complicated, but this remains a limitation of engines. Clearance is a user authentication solution for Rails applications and highlights the ongoing challenge within Rails regarding a standardized approach to user authentication.

00:05:03.240 Indeed, we have yet to settle on a convention for user authentication. Many developers reinvent the wheel, with plugins like restful authentication duplicating code within your application. Clearance aims to push this further, minimizing code within your Rails application while still being a full-fledged engine that provides controllers and views for common user authentication tasks, such as sign-in and sign-up.

00:05:18.660 For this talk, I created my own engine called Queso, which serves as a dynamic search engine for a database model. You can declare a database model as searchable, and Queso will provide a user interface to construct a query for that model and retrieve matching data rows. Queso’s structure looks similar to any standard Rails application, containing controllers, helpers, models, and views.

00:05:37.020 This is how Queso integrates into your application. I have a user model that I want to make searchable, so I mark it as 'Queso searchable' and provide options to configure its features. In the controller, I simply include the helper, which provides methods used in the view. In the view, I display the filter and results areas for that particular model.

00:06:00.660 Here’s an example of what the interface looks like: at the top, you have a form to construct a filter or query, where you can add constraints and sorting expressions before pressing the search button. Although this is not production-ready, it represents an initial foundation with potential for further development.

00:06:14.400 Ultimately, the only code in my application relates to these four lines on how Queso provides full MVC stack functionality. In summary, I've provided a quick walk-through of engines and their history, reflecting that engines are the next logical step toward making plugins more functional.

00:06:28.440 Initially, Rails applications lacked the ability to share code. With the introduction of plugins, developers could share Ruby code easily. Now, engines have expanded this capability, allowing not just Ruby code sharing but also sharing complete MVC functionality.

00:06:46.120 And that's all for my presentation. Any questions?

00:06:48.540 Yes, do you have any trouble testing in the context of using engines? That’s an insightful question. You need a framework to run your code. In my testing, I developed an example application based on the user model and conducted tests manually. I wouldn’t claim to be an expert in engine testing. Thoughtbot, with their Clearance engine, excels in testing, and they likely have extensive infrastructure offering valuable insights for best practices.

00:07:23.040 When it comes to your application, can you modify controllers or not? While you can copy or monkey patch code to make changes, your application’s view templates will always override those in the engine. If you require a different style or fix, simply copy the engine’s view template into your own application.

00:08:24.840 Engines have been a relevant concept for a number of years. Meta plugins offered engine-like features in older versions of Rails, such as 1.x. If needed, you can look into those options, but compatibility between those plugins and the official engine functionality in 2.3 could be challenging.

00:08:47.700 Is there a way for engines to be standalone applications? An engine is practically a Rails application; however, it may lack the full infrastructure usually seen in Rails applications, such as the script directory for running the server or console. Engines can serve as lightweight layers to expose their functionality, which can be handy for testing or demoing.

00:09:08.520 Unfortunately, creating a gem immediately tailored for launching an engine as an application without additional setup isn’t straightforward. It's a challenge to manage.

00:09:29.760 To conclude, can an engine depend on a plugin? I doubt it’s possible, as this type of recursion might not work properly. It poses an intriguing concept, but I doubt we would reach that level of nesting.

00:09:48.600 Engines have an exciting future ahead, and I'm eager to see how we can explore these concepts further. If there are no more questions, thank you for your time.

LoneStarRuby Conf 2009