00:00:16.480
All right, good afternoon everyone. It's barely afternoon, I guess.
00:00:21.680
So we're going to get started now. My name is Patrick Peak, and today I'm going to be talking to you about engines.
00:00:28.080
I'm always curious to see where folks are coming at this topic from, so a quick show of hands: who here has written their own engine? Go ahead and raise your hand.
00:00:39.920
All right. Who here has used an engine written by somebody else on a project? All right, a few more hands. And who here has never heard of engines and is here to figure out what they are?
00:00:46.719
All right, that's right. You guys are honest. That's great! Hopefully, there will be a little bit in this talk for everyone in all those groups.
00:00:59.840
A couple of other points: most of the code samples that I'm using today are written for Rails 4. However, not much has changed between Rails 3.2 and Rails 4, so you can use these samples pretty much as they are.
00:01:10.880
Hopefully, I'll have time for questions at the end, and I'll be hanging out here after lunch to answer them as well.
00:01:17.040
A little about me: I'm the Chief Technology Officer for Browser Media. We're an interactive web agency in Washington, DC, where we build websites for clients.
00:01:24.159
I'm also the project lead for BrowserCMS, which is an open-source content management system built on Rails that we frequently use to build those sites. BrowserCMS, as it happens, is an engine. Engines are exactly the sort of thing that make building tools like content management systems easier.
00:01:37.040
I want to start first with a problem. As our friend from Dilbert illustrates, sometimes repeating yourself is a little too easy when you're writing a Rails application.
00:01:49.280
You’re probably creating duplication even if your current application is completely DRY. If you look at any two apps you've written, there’s probably a good amount of duplication between those two apps.
00:02:00.000
As we should know in programming, duplication is the root of all evil. In Rails projects, we have historically had a number of ways to avoid repeating ourselves. The first option is to create a library if you need to reuse some code, especially if it's not Rails-specific.
00:02:26.560
You can make a library of it and distribute it as a gem, which is most appropriate if the code you're working with does not have any user interface components, like an XML parser or a testing library.
00:02:45.680
But what if you want something that's Rails-specific? For a long time, the go-to solution was plugins. Plugins could be added directly into projects into the vendor plugins directory, and later they were designed to be packaged as gems.
00:03:02.319
However, they were limited in what they could do. As of Rails 4, though, plugins, as you know them, are dead. And thank God for that! It's about time—if you have code in your projects under Rails 3, it will pester you insistently about deleting that.
00:03:21.760
So what other options do you have? Generators are another pretty common solution that folks will use. Rails has a few, and typically plugins would add some more of their own. These feel super productive, until the point where there's a bug in your generated code.
00:03:39.280
The more generated code you have in your project, the more costly that upgrade process becomes. For example, who here has ever used Restful Authentication on a project?
00:03:51.920
Keep your hand up if you ever went back and upgraded that code to a newer version of Restful Authentication. So, like one or two brave souls in the room. That illustrates the problem with generators: once you have all this code in your application, it's yours. You own it; it's not living in a gem elsewhere that you can just update.
00:04:09.760
So, generators are not going to be good enough. So what to do? As you may have guessed from the title of this talk, engines are our solution. The goal for today is to take a little bit of time and go through how we can better leverage these and take advantage of them.
00:04:20.880
Here’s the agenda I want to cover today: I'm going to talk about what engines are and why I care about them. We’re going to walk through a sample of building an engine, look at some interesting things you can do with engines, and finally discuss how we can get engines to work together.
00:04:44.720
Why do I care about engines? I've been working with content management systems for a long time. About 2003, we started building sites for clients, and there were no viable open-source CMS options available at the time. Like any eager young programmer, I figured I could knock out a CMS in a weekend. Totally easy, right? Ten years later, I'm still working on the same project.
00:05:27.200
The first version of BrowserCMS was written in Java, which had a big set of limitations on component reusability. We rewrote it in 2008 on Rails 2, around the same time that vendor plugins were falling out of favor.
00:05:52.160
We wrote it as a gem, but we still had to monkey patch Rails extensively. Every time you upgrade Rails, the entire thing would break.
00:06:09.759
Then in Rails 3, engines were introduced into the core for the first time. This was incredibly beneficial. The upgrade process was painful, as anyone who has done a Rails 2 to Rails 3 upgrade can attest, but it had a nice payoff.
00:06:30.160
When Rails 3 came out, they did a complete overhaul of engines, adding features that let us isolate our engines from application code and take advantage of features like the asset pipeline.
00:06:57.440
In the next release we’ll upgrade to 4.0, and I’m confident this will be seamless because most of the stuff now is dependent on these engine APIs, which are stable public APIs.
00:07:11.520
So what are engines? The short answer is, it’s a bit like a mini Rails application that you can add to other applications. Since it's packaged as a gem, you can easily reuse engines between projects.
00:07:25.440
The longer answer is: suppose you have a giant Rails application. I'm sure nobody other than me has one of those right now! Your business domain is what your client is paying you to build, but you’ve got a bunch of other stuff in there too—maybe you're selling stuff, managing users, or doing some blogging.
00:07:59.840
All of those might be useful for a second project, and you don’t want to be copying that blogging code out of your first app into your second app just to avoid rewriting it. With engines, we can split the behavior of this massive app into separate gems.
00:08:18.640
Instead of this one massive app, we can have a smaller focused app that concentrates on the business domain, and then just have it depend on and configure these external engines.
00:08:39.680
Now we have another app, and we want blogging; we can just have it also use those engines. Here are some popular engines that folks may be familiar with, showcasing what you can do with these tools.
00:09:10.160
The first is Spree, an e-commerce shopping cart solution built as an engine. It has all the standard stuff you would expect from a store, like product and order management. I don’t personally have much experience with it, but the Spree team is primarily based out of DC, and I know who to pester with questions.
00:09:51.360
Another popular engine is Devise, which is perhaps the most popular authentication solution for Rails right now. It provides a flexible way to handle user management and authentication. It’s very pluggable, allowing everything from email configuration to remember-me functionality.
00:10:30.640
A common use for engines is general-purpose admin dashboards, like Rails Admin. These allow for general exploration around the models in your application, generating a stock UI and dashboard that enables management of models. This can be a low-cost way to kickstart an admin area, similar to how Django provides an auto-generated management interface.
00:11:23.439
Now, let’s talk about building an engine. We'll work through an example of how to create your own engine and demonstrate some interesting functionalities within it. Suppose we decide what the world really needs is another content management system. It probably doesn't, but let's use that for our example.
00:12:12.400
To make it easier for other developers to add this to their projects, we're going to build it as an engine. I'm also super original with my project name and we'll call it Rubypress. Here is the command to generate a new mountable engine. And yes, I know that the command is called 'plugin,' and you may have heard somewhere that plugins are dead, but just trust me that’s what the command is actually called.
00:12:43.840
A quick digression: There are actually two sorts of engines: full and mountable. I'm not really going to cover full engines for today's discussion, as they don’t namespace things—making them problematic for gem distribution—but I'm happy to discuss that afterward.
00:13:11.440
When we run this command, we get a pile of code that looks quite similar to what you would get when creating a new Rails application. A couple of key points: you’re going to get a gemspec because we’re going to distribute this as a gem, an app directory like any Rails application, and a dummy Rails application placed in the test directory, which is automatically set up to use our engine.
00:13:55.680
The key takeaway here is that engines can’t live on their own; they need a Rails application to host and test them. The plugin command will generate that into your engine for you. So, for any content management system, we need a way to manage pages.
00:14:26.640
We'll add a resource to our engine, using the Rails generator command to give it a few fields. I’m using Rails 3.2 shorthand to implicitly get 'name,' 'path,' and 'content' as strings.
00:14:58.000
If you look at the controller file that was generated, in a normal Rails application that’s what you’d see. Since we’re working with a mountable engine, you’ll notice two changes: the controller is in a subdirectory called Rubypress and is namespaced under Rubypress.
00:15:25.760
Why is that? Because we created a mountable engine, everything gets isolated under a namespace. The 'engine.rb' file is the central configuration point for engines, and since we created an engine called Rubypress, all our classes fall under that namespace.
00:15:50.720
The engine.rb file will also automatically set up routes and generators to include that namespace in any generated files. Now, let’s talk about routing.
00:16:20.000
Since engines have their own controllers, you may wonder how to avoid conflicts. Your app’s routes will avoid conflicting with the engine’s routes. Rails solves this conflict by essentially having the engine be its own bucket of controllers.
00:16:51.440
Routes from one engine don’t mix with each other or the main application. The routes file in our engine project is similar to your project’s routes file, with the exception that the routes are being added to the engine class, rather than the application object.
00:17:27.040
How would we access our newly defined controller? Here’s the UI for our application. Assuming we add a bit of standard CRUD behavior to our pages controller, we would access the index action using the appropriate URL. The key thing to note is that we’ve nested this with a '/rubypress' in the URL.
00:17:53.680
Where does that come from? It’s defined in the application routes file, where we mount the engine using the mount method and specify the path for the engine.
00:18:19.760
You can also see we have another root route defined here, and the takeaway is that each engine and application can have their own separate route that’s accessible.
00:18:54.080
Once the routes are defined, let’s discuss linking to them. Both the engine and the main application have their own helper that hold collections to the routes for that engine. The main app routes will allow you to access the main application’s routes.
00:19:27.280
If you want to access the engine’s routes, you use this 'rubypress' helper. Accessing the root URL resolves to '/rubypress' — and then, for standard CRUD access, we can pass a page object to the 'pages' path in the Rubypress engine.
00:20:03.760
There’s a bit of a hassle with prefixing all our routes this way; fortunately, Rails can implicitly decide which set of routes you want based on the file you're in.
00:20:16.760
In the layouts file from our engine, we can create a menu that accesses both main application and engine routes without requiring any prefixes for engine routes.
00:20:44.800
If we want to access the main application, we do need to specify the helper method. If we flip this around to look within the main application itself, our application routes don't need a prefix, but we do need them for accessing Rubypress paths.
00:21:10.760
Now, where does that 'rubypress' helper come from? True to Rails’ convention over configuration approach, Rails automatically generates this based on the engine's name. If you want to change this name, you can do so in two places: in your project’s routes file or in the engine itself.
00:21:49.520
Let’s talk about migrations next. This is one of the significant additions introduced in Rails 3.1. Before then, migration management was fairly non-standard. This is the migration file from the resource we generated earlier, and you can see the namespacing showing up in two places: in the name of the file and in the name of the table.
00:22:58.400
The great thing is that the engine will namespace all of your tables, which reduces the chance for conflicts. Since this migration exists in the gem, the next step is to get it out of the gem into your project.
00:23:45.760
Engines will provide a rake task, and when you invoke it, it copies the migration files from the engine into your project. It won't override migrations that already exist with the same name.
00:24:09.440
This is beneficial when needing to upgrade and release a new version of your engine. Users can simply rerun the task, receiving only the new migrations without disrupting their old setup.
00:24:28.640
After running the command, this is the name of the migration file you'll get, with a timestamp reflecting when you added it to your application. This preserves the order of your migrations.
00:24:48.160
Additionally, it suffixes the name of the engine to the migration file, making it easy to see which migrations are from which engine. If you have several engines and want all migrations at once, there’s a rake task that will copy everything across.
00:25:08.320
Next, let’s talk about configuration. In a traditional Rails application, you might find configuration in either the environment file or in an initializer. You can do the same in an engine, allowing users to just require the gem and only change configurations if they don’t like the defaults.
00:25:47.760
Most of the standard Rails application configuration behavior is available from within an engine. The engine.rb file is very similar to the application.rb file. I encourage you to look at the Rails configuration guide and experiment with what you can do.
00:26:21.440
One thing you may want to do is add initializers. Engines can define their initializers that hook into the Rails boot-up sequence. Rails defines a series of hooks during startup where we can specify our initializers.
00:27:04.160
For instance, we might want to add additional DSL to the routing mapper. Here’s how we might define a new method in our engine project, allowing us to easily mount Rubypress at a reasonable default and adding another custom route.
00:27:39.760
You will need to namespace the controller name because this method gets invoked within the context of your application routes file. This example is somewhat trivial but demonstrates how to streamline adding things to a project.
00:28:21.280
You can use the same technique to augment any of the core Rails pieces, like ActiveRecord or ActiveController. Most of the core components in Rails are now simply Rails engines, which defines the configuration of how things fit together.
00:28:58.080
Engines can also define their middleware, so we could configure Rack Cache since we're a CMS that needs to cache pages. This middleware will automatically process any routes that get passed to our engine because engines also act as their own Rack endpoint.
00:29:35.040
With engines, you can pre-configure the generators a project will use. For example, if we want our spec files to be used everywhere instead of test unit whenever we run generators, we can simply include an engine that changes the project defaults.
00:30:13.920
Providing configuration options for engines greatly increases usability, as users will often want to make changes after adding your engine. When offering configuration options, it’s vital to set reasonable defaults.
00:30:51.680
For instance, we can create a configuration namespace for the Rubypress engine, set up reasonable defaults, and allow users to override them in their application’s environment files.
00:31:26.640
Now that we’ve created our engine, how do we get people to use it? The best way to facilitate this is by making it easy for new users to get started. At the very least, write some documentation that informs users where to start.
00:32:00.000
Update the README file to contain useful information, rather than just the default content it generates. GitHub will display the README prominently, so it’s a crucial place for users.
00:32:36.640
Document the exact steps needed to install your project. By listing the steps, you can see if there might be too many, which is a cue for simplification. Ideally, we want the installation process to be as seamless as possible.
00:33:00.320
By leveraging generators, engines can bundle installation scripts that automate more of the setup process. This gives us a powerful tool to streamline user onboarding.
00:33:25.440
Now, let’s talk about the asset pipeline. One of the major changes introduced in Rails 3.1 was the asset pipeline, which many developers are familiar with today. The asset pipeline allows us to organize and compress JavaScript, HTML, CSS, and images.
00:34:01.040
Importantly for our purposes, the asset pipeline allows us to package view-related code in the engine in a logical manner. Previously, managing view-related assets was tedious.
00:34:23.280
For example, if you had a gem with complex JavaScript code and HTML, such as CKEditor, you would previously need to write a Rake task to copy those files into the public directory of your project.
00:34:56.000
If changes needed to be made, it would be your responsibility to manage those files. With the asset pipeline, engines can package and provide all view-related code within the engine, avoiding the need for duplication.
00:35:29.920
For instance, the CKEditor Rails gem operates solely as an engine with HTML, JavaScript, and CSS. No Ruby code is needed, allowing for more straightforward management of packages.
00:36:09.280
This pattern of utilizing pure front-end engines makes it easier for developers to utilize various reusable components without worrying about duplicating code.
00:36:27.200
As someone who prefers to delete unnecessary code, the asset pipeline has been a blessing, enabling me to clean up a significant amount of redundant files from the BrowserCMS project.
00:37:00.000
Now let’s discuss how engines can and can’t work together. There are several challenges, but because engines are a relatively new concept, we have yet to fully develop optimal patterns for this integration.
00:37:29.440
The most straightforward use cases arise when you have a set of engines designed to work together. For example, using BrowserCMS alongside additional engines for news or blogging makes perfect sense.
00:38:01.920
However, it’s not as simple as just dropping in an unrelated engine, such as one from Radiant, since they haven’t been designed with compatibility in mind.
00:38:32.640
One advantage of establishing a common stack is the ability to create conventions that related engines can follow. This yields a cohesive experience for both developers and users.
00:39:03.120
For instance, in BrowserCMS, all engines support a unified installer script, allowing us to quickly add engines to our projects with a single command.
00:39:36.320
However, several challenges arise when multiple engines are used together due to potential dependency conflicts.
00:39:59.760
Consider engines like Spree and Rails Admin that may have dependencies on different versions of libraries like Devise. Resolving these dependency conflicts may require forking gems or abandoning one of the engines.
00:40:22.520
When writing an engine, you can create a better experience by minimizing dependencies, following rational versioning, and being liberal with what your engine accepts as a dependency.
00:40:46.160
Next, there’s no common security model in Rails. Any non-trivial engine will likely need some form of authentication. For example, BrowserCMS uses Restful Authentication, while Spree has its own authentication approach.
00:41:08.800
The lack of a standard method complicates developing engines meant to work together, as there is no unifying authentication solution in our community. With a fragmented landscape, finding a common denominator is essential.
00:41:37.680
Lastly, mismatched user interfaces can cause other issues when adopting multiple engines. It’s rare for two UIs from different engines to be designed to work together seamlessly.
00:42:05.680
Two examples are Active Admin and Rails Admin, which are intended for similar purposes but offer different user interfaces that may not integrate well.
00:42:36.760
You can approach this by designing engines with UI-free functionality, leaving it to developers to build their UI according to your APIs, or by embracing a single UI paradigm.
00:43:06.640
To recap what I hope you learned from today’s talk, engines are the future, and they are powerful tools for Rails development.
00:43:36.480
When properly executed, engines make code reuse easy and can reduce errors stemming from duplication. Thank you for your time, and that’s the end of the talk.