00:00:12.049
Welcome everybody, and thanks for coming today. I'm going to talk a little about Rails and how awesome it is.
00:00:17.760
I think part of why it is so awesome, and why it's great, is its community and the huge ecosystem around it. When I try to explain Rails to newcomers to the industry or even my colleagues, I think of Rails like a city map—this huge mess of sitting rabbits. It's big and consists of different areas, like the Active Record district over there and Initializer Town.
00:00:31.380
In our day-to-day work, it can feel kind of intimidating—sometimes you feel surrounded by all this functionality and you don't know what's happening. You feel constrained.
00:00:38.969
While it's true that Rails has certain ways and patterns about how things should be done, this does not mean we cannot find or create our own little areas of freedom.
00:00:49.500
Today, I'm going to talk about some of the areas we explore to help us work more efficiently with Rails. At the beginning, I want to talk a little bit about the status quo, then we'll discuss breaking some conventions, followed by a quick wrap-up about general application architecture and some of the trade-offs.
00:01:05.159
So, quick question to the room: who here knows about the band Status Quo? A couple of hands, alright, cool!
00:01:10.830
In contrast to Rails, the band Status Quo had a very simplistic approach; they always only played with three chords. But I'm not going to continue today to talk about the band Status Quo—coming here from London, I should at least pick up on a British reference.
00:01:21.680
Now, moving on to the actual status quo as in the current state of affairs. Two years ago, at RailsConf, I mentioned that Rails for me is like my proper pack.
00:01:27.980
If the world ends tomorrow, I still want to be able to use Rails to write web applications. I think this is a really cool feature and idea behind it, because everything comes with Rails—like batteries included.
00:01:38.969
You can start writing web applications without the need for anything else. I'm more of an ultralight backpacking person myself.
00:01:49.730
While I'm not prepared for the zombie apocalypse, I always have just enough gear with me for my current adventure. This reflects the Pareto principle, commonly known as the 80/20 rule.
00:02:01.979
For example, 80% of users use only 20% of your product's functionality, or in Rails' case, 80% of applications out there only utilize 20% of what Rails offers.
00:02:09.420
The background for this talk relates to the Omakase Rails applications. Throughout several years, having completed a couple of Rails projects, we felt the same pain over and over again: that Omakase Rails applications often lead to slower performance.
00:02:24.650
This is a problem for us at Flight, because we are used to quick TDD cycles. We want to be able to write a little code, run the tests, and get immediate feedback.
00:02:36.680
This process of writing a little code and running the test happens, I don't know, five or six times a minute—sometimes if I'm typing poorly, ten times! I want this fast feedback.
00:02:49.160
Unfortunately, as a Rails codebase grows, if you follow the regular Omakase style, you lose out on these quick feedback cycles.
00:03:02.810
In a previous application, where we felt this pain, we built what will serve as a reference throughout this talk. We ended the project with around 4,800 tests, and the full test suite ran in six minutes.
00:03:17.269
Waiting six minutes to determine whether a refactor worked or not is too long, in my opinion. I need my codebase to be malleable in order to make quick decisions.
00:03:29.910
I want to know quickly if the refactoring worked, if this bug is fixed, or if it's not, and I want to get that feedback rapidly.
00:03:43.669
After seeing this issue arise in a couple of projects, we went back to the drawing board, put on our thinking hats, and really dived into what the actual issue was.
00:03:56.340
One hotspot we identified was the Rails boot process, which looks roughly like this: at the beginning, the Rails framework is loaded.
00:04:02.810
Next, it loads all dependencies from your Gemfile, then it initiates its initialization process, which pulls in all the code from all configured writers, engines, and so on.
00:04:11.150
During this process, something called eager loading kicks in, which is responsible for acquiring all the code inside your app directory.
00:04:24.560
We started to modify our configuration a little bit. In a typical Rails application, you see a line in your application.rb file like this: 'require Rails all.'
00:04:38.620
We changed that to require only the two frameworks we needed for this particular application: Action Controller and Sprockets for asset management.
00:04:55.230
Once we made this change, we learned that there's already support for this kind of configuration by Rails if you run 'rails new' with the --help flag.
00:05:10.420
You get a list of all configuration options you can provide, and it actually allows you to skip a couple of plugins already. You see options to skip Active Record and Action Cable.
00:05:25.360
If you don't need them, you can already tell Rails not to generate a new project with these dependencies. I did run this command with the --skip test flag because I usually use RSpec for testing.
00:05:38.490
What it generates is an application.rb that no longer requires Rails all; it only requires core Rails, which is the bare minimum.
00:05:54.370
Then, it tells you to pick the frameworks you want, and it commented out the test unit Rails side already for me because I was skipping that in the previous step.
00:06:10.300
After requiring the necessary plugins, there's another line that looks like this: 'require Rails groups.' In previous versions of Rails, it looked a little different.
00:06:24.660
It originally required the default and then the Rails end. This hasn't changed, but it instructs Bundler to require all gems from your Gemfile for your current environment.
00:06:36.850
The problem with this line is that it will add a linear load time to your boot process, which means that the more gems you add, the longer it will take when you load your application.
00:06:49.340
Unfortunately, this is a hard fact we need to accept; there is no way around it.
00:07:01.610
We continued tweaking a couple more settings. One of them is an Active Support feature that allows you to specify a flag to say, 'Hey, don't load everything from Active Support but only the dependencies I actually need to boot up Rails.'
00:07:14.880
There are two more settings: cache classes, which keeps a loaded class cached, and we specifically disabled the dependency loading.
00:07:22.490
This was because at that time we were big fans of something called Screaming Architecture, which I will cover later in the presentation.
00:07:34.109
As a quick heads up, the idea of Screaming Architecture is that it should be obvious for a new team member to discern the type of application just by looking at the directory structure.
00:07:50.120
If we take a Rails codebase as an example, when you generate a new Rails codebase, you see a bin, config, lib, public, and many directories without revealing the purpose of this application.
00:08:02.580
We started moving everything from the app directory into the lib directory with a proper namespace. For instance, if the application was a movie organization app, we moved everything into a namespace.
00:08:17.090
So, you would see an edit, index, and show ERB template, a movie model, movie repository, along with the movie controller. Everything that was in app was moved to lib.
00:08:31.620
We deleted the app directory altogether, ensuring that nothing would be loaded unless we required it explicitly.
00:08:41.200
To enforce this, we adjusted a few more settings in the application.rb. We made sure to set 'eager load false' so that nothing would be loaded.
00:08:52.170
We also made sure not to look up any parts—essentially emptying them manually.
00:09:08.690
All was well until here.
00:09:11.830
We booted up our application with no errors. However, when trying to hit the next page, it blew up with an exception: 'uninitialized constant MoviesController.'
00:09:22.990
As I mentioned before, Rails expects everything to be loaded upfront—that's why eager loading happens in the first place.
00:09:35.800
So now Rails couldn't find our application controller anymore. Luckily, there is something called lazy loading to the rescue.
00:09:48.500
In this context, you can see a call to Active Support's 'constantize.' At that time, Rails tried to convert the controller name based on the route definition we provided.
00:10:01.600
Because Ruby is flexible and awesome, we monkey-patched the 'constantize' method, allowing it to also require the controller if it was not already loaded.
00:10:11.170
This allowed everything to work—we could serve requests, and everything functioned fine.
00:10:22.250
But it had an interesting effect on our codebase: now we needed to have explicit requires everywhere.
00:10:34.880
This was a contentious decision within the team, but I found it helped me see how many dependencies my current file had.
00:10:42.210
I refer to this as manual static analysis. If you open up a file with 20 require statements, at least it hints that maybe this file is doing too much.
00:10:54.080
Then you can start thinking about extracting or splitting things up in different ways. Maybe it's okay, but at least you can start having conversations about the structure.
00:11:07.040
With all these settings configured, we were able to start a new Greenfield project, which serves as our example.
00:11:18.720
In this project, everything was at our disposal; we could do whatever we wanted because we were in complete control of the setup.
00:11:40.820
Here's the previous application again, with a six-minute runtime for the entire test suite. I wasn't part of the project when it kicked off, but I joined after it had around 5,500 tests that ran in 23 seconds.
00:11:58.090
With all these tweaks, loading or testing one controller in isolation used to take around 18 seconds.
00:12:01.920
Now, with a new approach and configuration settings, it only took two seconds, which I think is pretty good.
00:12:13.540
One caveat is that we made a conscious decision to not use Active Record at all; we switched to the Sequel gem and a repository pattern.
00:12:24.400
We didn't need to discard Active Record for that; it's mostly about being able to switch our implementations for persistence depending on the environment.
00:12:41.670
So, when we were testing, we had an in-memory repository in order to gain more speed, and in production, we used a real repository connected to a Postgres database.
00:12:55.440
It’s worth mentioning that this introduced a double burden on us, since we needed to maintain two separate implementations for each repository.
00:13:07.760
We had to maintain both the Postgres implementation and the in-memory implementation. The benefits of this approach were that we gained a lot of speed for our test suites.
00:13:28.810
It was worth investing in maintaining this double structure. We did this with shared state examples that ran for both implementations.
00:13:40.160
This meant we only had to write the test once, but implement both versions.
00:13:52.080
The second example I have is a Rails application we took over from a new client. This application started with 220 gems and a liberal usage of Active Record.
00:14:02.700
The application came with 2,600 tests, of which 730 were controller tests, and the rest included various types of tests, like Capybara and Cucumber tests.
00:14:14.390
The entire test suite for this application ran in around eight and a half minutes.
00:14:28.560
We thought, okay, let's optimize it our way to gain more speed. Running a single controller with the existing application, which required Rails, always took around 14 seconds.
00:14:41.330
After making all the configuration changes, we managed to cut it down to nine seconds, which felt somewhat better, but it wasn't as efficient as before.
00:14:54.540
For a non-controller spec, it looked a little better. Running a single non-controller spec used to take around six and a half seconds.
00:15:05.400
We were able to cut this down to around half a second, providing a better optimization than with controller tests.
00:15:20.210
However, we couldn't just apply this optimization across the entire codebase because manually adding all required statements took a lot of effort.
00:15:33.880
I did all the benchmarking for two controllers, and by the end, I had my good status show me that I had 150 tests to run.
00:15:44.060
Since this was a client project, we couldn't make breaking refactors just for the sake of refactoring.
00:15:56.580
Instead, we needed to find ways to slip these changes into new features as we added them.
00:16:10.000
So, we couldn't simply flip the switch and say, 'Yes, now your application is faster with a better development experience.' However, we projected that we could cut down the runtime from eight and a half minutes to five or six minutes.
00:16:22.210
This was still too long for my taste, but it was an improvement.
00:16:34.090
With all the changes and explicit requires, not many people would agree with my point.
00:16:48.360
The main point I want to make here is that launching a test suite shouldn't be a daunting task. You should be able to run your tests multiple times a minute to get fast feedback.
00:17:04.060
If you take away just one thing from this talk, it's this: split up your spec helpers. Create a spec helper that loads your specs and defines global test covers that you want to use.
00:17:20.150
Have a separate Rails helper that requires the existing spec helpers and also requires the Rails environment. This includes the test speed benefit.
00:17:32.660
If you only test a class that doesn't need anything from Rails, you will experience a substantial speed benefit.
00:17:44.240
Now, let’s talk about what constitutes the 'Rails Way.' What does this actually mean?
00:17:58.000
In Rails doctrine, we can read that we value convention over configuration. We made a conscious decision as a team to favor explicitness over impressiveness.
00:18:12.780
I prefer to read explicit and boring code over magical code that makes assumptions. That doesn't mean I don't like Rails; I just favor explicit code.
00:18:26.760
It's worth mentioning that conventions are heuristics—they are not hard rules. You can break them.
00:18:42.220
If you learn something new, you might realize that conventions you've followed for three years may not apply anymore.
00:18:56.250
It's not a bad thing. The 80/20 rule applies here again; all these conventions in Rails work well for 80% of applications, but the remaining 20% might need something else.
00:19:09.150
That's perfectly fine! This flexibility is one of the strengths of Rails—you just might have to put in some extra work to capitalize on the benefits.
00:19:23.450
Earlier, I mentioned I want to talk about general architecture. I'm not going to bore you with outdated material on what software architecture should look like.
00:19:37.000
There are a couple of more recent architectures out there. First, there's Clean Architecture, which defines some entities at the core of the application, orchestrated by certain use case implementations.
00:19:53.920
These use cases are exposed through controllers in a Rails application, for example, through the web. Similarly, there's a hexagonal architecture, coined by Alistair Cockburn, that follows a similar direction.
00:20:05.780
At the core of your application, you have entities, and you can provide various adapters for different clients. You might have a GUI adapter, or a REST adapter that connects to Salesforce.
00:20:17.560
Last but not least, in Martin Fowler's book, "Patterns of Enterprise Application Architectures," there's the service layer—an idea that again illustrates you have a user interface connecting to a service layer that guards your domain model.
00:20:30.440
The service layer in turn has access to the datasource layer, which represents your database at its core.
00:20:44.800
If we look at these three architectures, they all share a similar idea: it's not something completely new. If we take the service layer as an example and zoom in, it resembles layered architecture.
00:20:56.670
Nothing new to learn; we're back in the past with new words for old concepts—essentially, isolating your application from the outside world.
00:21:09.500
It’s important to remember, though, that Rails is not your application. Business concerns should be paramount.
00:21:23.110
I want to show you an example using an e-commerce platform. Imagine by having a user class; when a user logs in, they represent a customer.
00:21:38.740
This user could have a basket, containing several products that each have categories. As we check out, the user selects a delivery method and a payment option.
00:21:54.260
At the end of the payment process, we get an invoice as a PDF download. This part is the core of our application—there is no controller present, no session hash, or any of the Rails specifics.
00:22:05.150
This aligns with the principles of Domain-Driven Design, a book I highly recommend.
00:22:18.860
To tie this back to the architecture, if we look at how this applies to our Rails app: we start from the top down. The user interface typically is the browser.
00:22:33.450
This browser connects through HTTP to our application, such as our checkout controller, which translates the web request to the checkout use case implementation.
00:22:49.970
This instance and checkout process mediates between several domain model objects, which should ideally just be plain old Ruby objects defining their own behavior.
00:23:01.500
Ultimately, this culminates in creating an order, which serves as our gateway to the database.
00:23:12.590
At its core, it's a straightforward process: a request comes in and interacts with something in the database. How does this link back to the MVC concept?
00:23:26.720
MVC has become more of an abstract concept than a strict design pattern, especially noted in the Rails community where the value of skinny controllers over fat models was discussed.
00:23:38.850
But we realized that these models became hard to test, leading to the opposite extreme of keeping the models thin and overloading controllers with behavior.
00:23:50.690
It isn't sufficient to merely consider these three roles; I like to envision it as a balloon.
00:23:59.880
If you squeeze one end of the balloon, you're not eliminating air or responsibilities; you're redistributing them to the other side.
00:24:13.000
It’s essential to not fixate on one aspect to the exclusion of others. Focus on keeping everything healthy.
00:24:25.770
By dispersing responsibilities into smaller, manageable pieces, we create a codebase that is much easier to understand.
00:24:39.450
If we only concentrate on classes with 5 to 10 lines of code, it becomes inherently easier to follow than wading through a 50-method long monolith.
00:24:53.070
While preparing for this talk, I had a conversation with a friend during coffee about the style we’ve practiced in Rails for years.
00:25:05.190
He asked me if I would do it again, if I would still implement my expectations in this way. I thought for a moment.
00:25:17.270
I agree with all the preloading and loading optimizations, but the concept of Screaming Architecture—removing the app directory and shifting everything?
00:25:30.240
Probably not. I think that idea is overrated. If you consider the default directory structure, you don't need to identify it as a movie organization or an e-commerce platform.
00:25:45.670
It's acceptable to start with something simple—like, 'This is a reservation system.' If I want to know more, I look into lib or source for the namespaced responsibilities.
00:26:01.080
With the separation of concerns, I like the idea of having these physically separated—my core application in lib, with nothing Rails-specific leaking in there.
00:26:19.790
Everything Rails and web-related should remain in app. This approach simplifies the upgrade process, as we wouldn't need to worry about anything in lib.
00:26:34.880
We might wrap some Active Job dependencies, but that’s about it. All of this comes with trade-offs, though.
00:26:46.770
I'm not trying to sell you promised land here—it's not that magically we'll all end up writing more awesome Rails apps.
00:27:00.290
There are trade-offs and considerable ones. For instance, explicit requires aren't commonly seen in typical Rails applications.
00:27:13.950
Carrying back to the city metaphor at the beginning: we put up some big construction sites and cut through several blocks.
00:27:27.370
It's not without thought, but we need to be aware of the cost—it requires consideration.
00:27:40.460
We work in teams and are not islands, so it needs to be a team decision; the team's effectiveness is more crucial than individual idealism.
00:27:54.500
After all, in the end, it’s a technical detail whether a file for a controller lives in this directory or that directory. Who cares?
00:28:06.750
I typically begin every project like this. If I were to start a new project now, I would wait until it becomes painfully slow before I worry about optimizing.
00:28:20.030
For instance, if I set up a Rails application for people to sign up for my birthday party, I would use the free tier of Heroku, scaffold it and deploy it, then call it a day.
00:28:36.890
It all comes down to how much maintenance you expect for this problem and how much benefit you gain from optimizations.
00:28:52.120
We should also remember to not just mindlessly follow traditions passed on to us; we need to question everything, break things, and fix them again.
00:29:10.110
Use the Leaning Tower of Pisa as an example—its attraction draws massive crowds every year.
00:29:26.270
Is that an indicator that we should build every tower like it? Probably not! And finally, know your tools.
00:29:40.780
Understand why you follow certain rules, but also know when to break them. Make conscious decisions.
00:29:54.400
On that note, I'd like to leave some room for questions, and first, I want to say thank you for listening to me.