Ruby Video
Talks
Breaking Bad - What Happens When You Defy Conventions?
Christoph Gockel
1 talk
#ruby-on-rails
#performance
#test-driven-development
#software-architecture
#refactoring
#activesupport
Summarized using AI

Breaking Bad - What Happens When You Defy Conventions?

by Christoph Gockel

In the presentation titled "Breaking Bad - What Happens When You Defy Conventions?" delivered by Christoph Gockel at RailsConf 2017, the speaker explores the conventional structures within Ruby on Rails applications and discusses the implications of deviating from these norms. Rails, celebrated for its comprehensive convention, sometimes imposes limitations that do not suit all applications. The speaker encourages developers to examine their practices and consider the benefits of customizing their Rails applications.

Key points discussed include:
- Understanding the Status Quo: Gockel highlights the established effectiveness of Rails, likening the framework to a complex city map with various districts. While Rails offers valuable functionality, some developers may feel constrained by its strict conventions.
- Identifying Issues with Traditional Rails Applications: The presenter shares experiences from various projects, emphasizing how standard Rails practices led to longer testing cycles and delayed feedback. He mentions the Pareto principle, noting that many users only utilize a fraction of Rails’ extensive features.
- Refactoring for Efficiency: Gockel explains the steps taken to optimize their testing speed by modifying Rails configurations. By skipping unnecessary initializations and employing lazy loading techniques, they managed to significantly reduce the boot time of their applications.
- Case Studies: The speaker presents two main examples: a newly designed Greenfield project that implemented aggressive optimizations leading to a dramatic decrease in testing runtime from six minutes to 23 seconds and a legacy application where improvements were more challenging but still achieved reductions in testing times.
- Emphasizing Explicitness over Implicitness: A central tenet of Gockel’s approach is the decision to favor explicit code. The speaker suggests moving from the default Rails directory structure to a clearer layout that prevents Rails dependencies from leaking into core application code.
- Alternative Architectural Patterns: Gockel introduces ideas from modern software architecture, including clean and hexagonal architectures, advocating for structures that isolate the application logic from Rails specifics while ensuring testability and maintainability.
- Final Thoughts: The talk concludes with a reminder that conventions are not immutable rules. Developers should feel empowered to challenge norms when their application demands different strategies. Flexibility in design can optimize performance, but it is crucial to make these decisions as a team, focusing on overall productivity rather than individual preferences.

Participating in this session leads to vital insights about how to harness the flexibility of Rails, emphasizing that understanding when to follow conventions and when to break them ultimately enhances the development process.

00:00:12.049 Welcome everybody, and thanks for coming today. I'm going to talk a little about Rails and how awesome it is.
00:00:17.760 I think part of why it is so awesome, and why it's great, is its community and the huge ecosystem around it. When I try to explain Rails to newcomers to the industry or even my colleagues, I think of Rails like a city map—this huge mess of sitting rabbits. It's big and consists of different areas, like the Active Record district over there and Initializer Town.
00:00:31.380 In our day-to-day work, it can feel kind of intimidating—sometimes you feel surrounded by all this functionality and you don't know what's happening. You feel constrained.
00:00:38.969 While it's true that Rails has certain ways and patterns about how things should be done, this does not mean we cannot find or create our own little areas of freedom.
00:00:49.500 Today, I'm going to talk about some of the areas we explore to help us work more efficiently with Rails. At the beginning, I want to talk a little bit about the status quo, then we'll discuss breaking some conventions, followed by a quick wrap-up about general application architecture and some of the trade-offs.
00:01:05.159 So, quick question to the room: who here knows about the band Status Quo? A couple of hands, alright, cool!
00:01:10.830 In contrast to Rails, the band Status Quo had a very simplistic approach; they always only played with three chords. But I'm not going to continue today to talk about the band Status Quo—coming here from London, I should at least pick up on a British reference.
00:01:21.680 Now, moving on to the actual status quo as in the current state of affairs. Two years ago, at RailsConf, I mentioned that Rails for me is like my proper pack.
00:01:27.980 If the world ends tomorrow, I still want to be able to use Rails to write web applications. I think this is a really cool feature and idea behind it, because everything comes with Rails—like batteries included.
00:01:38.969 You can start writing web applications without the need for anything else. I'm more of an ultralight backpacking person myself.
00:01:49.730 While I'm not prepared for the zombie apocalypse, I always have just enough gear with me for my current adventure. This reflects the Pareto principle, commonly known as the 80/20 rule.
00:02:01.979 For example, 80% of users use only 20% of your product's functionality, or in Rails' case, 80% of applications out there only utilize 20% of what Rails offers.
00:02:09.420 The background for this talk relates to the Omakase Rails applications. Throughout several years, having completed a couple of Rails projects, we felt the same pain over and over again: that Omakase Rails applications often lead to slower performance.
00:02:24.650 This is a problem for us at Flight, because we are used to quick TDD cycles. We want to be able to write a little code, run the tests, and get immediate feedback.
00:02:36.680 This process of writing a little code and running the test happens, I don't know, five or six times a minute—sometimes if I'm typing poorly, ten times! I want this fast feedback.
00:02:49.160 Unfortunately, as a Rails codebase grows, if you follow the regular Omakase style, you lose out on these quick feedback cycles.
00:03:02.810 In a previous application, where we felt this pain, we built what will serve as a reference throughout this talk. We ended the project with around 4,800 tests, and the full test suite ran in six minutes.
00:03:17.269 Waiting six minutes to determine whether a refactor worked or not is too long, in my opinion. I need my codebase to be malleable in order to make quick decisions.
00:03:29.910 I want to know quickly if the refactoring worked, if this bug is fixed, or if it's not, and I want to get that feedback rapidly.
00:03:43.669 After seeing this issue arise in a couple of projects, we went back to the drawing board, put on our thinking hats, and really dived into what the actual issue was.
00:03:56.340 One hotspot we identified was the Rails boot process, which looks roughly like this: at the beginning, the Rails framework is loaded.
00:04:02.810 Next, it loads all dependencies from your Gemfile, then it initiates its initialization process, which pulls in all the code from all configured writers, engines, and so on.
00:04:11.150 During this process, something called eager loading kicks in, which is responsible for acquiring all the code inside your app directory.
00:04:24.560 We started to modify our configuration a little bit. In a typical Rails application, you see a line in your application.rb file like this: 'require Rails all.'
00:04:38.620 We changed that to require only the two frameworks we needed for this particular application: Action Controller and Sprockets for asset management.
00:04:55.230 Once we made this change, we learned that there's already support for this kind of configuration by Rails if you run 'rails new' with the --help flag.
00:05:10.420 You get a list of all configuration options you can provide, and it actually allows you to skip a couple of plugins already. You see options to skip Active Record and Action Cable.
00:05:25.360 If you don't need them, you can already tell Rails not to generate a new project with these dependencies. I did run this command with the --skip test flag because I usually use RSpec for testing.
00:05:38.490 What it generates is an application.rb that no longer requires Rails all; it only requires core Rails, which is the bare minimum.
00:05:54.370 Then, it tells you to pick the frameworks you want, and it commented out the test unit Rails side already for me because I was skipping that in the previous step.
00:06:10.300 After requiring the necessary plugins, there's another line that looks like this: 'require Rails groups.' In previous versions of Rails, it looked a little different.
00:06:24.660 It originally required the default and then the Rails end. This hasn't changed, but it instructs Bundler to require all gems from your Gemfile for your current environment.
00:06:36.850 The problem with this line is that it will add a linear load time to your boot process, which means that the more gems you add, the longer it will take when you load your application.
00:06:49.340 Unfortunately, this is a hard fact we need to accept; there is no way around it.
00:07:01.610 We continued tweaking a couple more settings. One of them is an Active Support feature that allows you to specify a flag to say, 'Hey, don't load everything from Active Support but only the dependencies I actually need to boot up Rails.'
00:07:14.880 There are two more settings: cache classes, which keeps a loaded class cached, and we specifically disabled the dependency loading.
00:07:22.490 This was because at that time we were big fans of something called Screaming Architecture, which I will cover later in the presentation.
00:07:34.109 As a quick heads up, the idea of Screaming Architecture is that it should be obvious for a new team member to discern the type of application just by looking at the directory structure.
00:07:50.120 If we take a Rails codebase as an example, when you generate a new Rails codebase, you see a bin, config, lib, public, and many directories without revealing the purpose of this application.
00:08:02.580 We started moving everything from the app directory into the lib directory with a proper namespace. For instance, if the application was a movie organization app, we moved everything into a namespace.
00:08:17.090 So, you would see an edit, index, and show ERB template, a movie model, movie repository, along with the movie controller. Everything that was in app was moved to lib.
00:08:31.620 We deleted the app directory altogether, ensuring that nothing would be loaded unless we required it explicitly.
00:08:41.200 To enforce this, we adjusted a few more settings in the application.rb. We made sure to set 'eager load false' so that nothing would be loaded.
00:08:52.170 We also made sure not to look up any parts—essentially emptying them manually.
00:09:08.690 All was well until here.
00:09:11.830 We booted up our application with no errors. However, when trying to hit the next page, it blew up with an exception: 'uninitialized constant MoviesController.'
00:09:22.990 As I mentioned before, Rails expects everything to be loaded upfront—that's why eager loading happens in the first place.
00:09:35.800 So now Rails couldn't find our application controller anymore. Luckily, there is something called lazy loading to the rescue.
00:09:48.500 In this context, you can see a call to Active Support's 'constantize.' At that time, Rails tried to convert the controller name based on the route definition we provided.
00:10:01.600 Because Ruby is flexible and awesome, we monkey-patched the 'constantize' method, allowing it to also require the controller if it was not already loaded.
00:10:11.170 This allowed everything to work—we could serve requests, and everything functioned fine.
00:10:22.250 But it had an interesting effect on our codebase: now we needed to have explicit requires everywhere.
00:10:34.880 This was a contentious decision within the team, but I found it helped me see how many dependencies my current file had.
00:10:42.210 I refer to this as manual static analysis. If you open up a file with 20 require statements, at least it hints that maybe this file is doing too much.
00:10:54.080 Then you can start thinking about extracting or splitting things up in different ways. Maybe it's okay, but at least you can start having conversations about the structure.
00:11:07.040 With all these settings configured, we were able to start a new Greenfield project, which serves as our example.
00:11:18.720 In this project, everything was at our disposal; we could do whatever we wanted because we were in complete control of the setup.
00:11:40.820 Here's the previous application again, with a six-minute runtime for the entire test suite. I wasn't part of the project when it kicked off, but I joined after it had around 5,500 tests that ran in 23 seconds.
00:11:58.090 With all these tweaks, loading or testing one controller in isolation used to take around 18 seconds.
00:12:01.920 Now, with a new approach and configuration settings, it only took two seconds, which I think is pretty good.
00:12:13.540 One caveat is that we made a conscious decision to not use Active Record at all; we switched to the Sequel gem and a repository pattern.
00:12:24.400 We didn't need to discard Active Record for that; it's mostly about being able to switch our implementations for persistence depending on the environment.
00:12:41.670 So, when we were testing, we had an in-memory repository in order to gain more speed, and in production, we used a real repository connected to a Postgres database.
00:12:55.440 It’s worth mentioning that this introduced a double burden on us, since we needed to maintain two separate implementations for each repository.
00:13:07.760 We had to maintain both the Postgres implementation and the in-memory implementation. The benefits of this approach were that we gained a lot of speed for our test suites.
00:13:28.810 It was worth investing in maintaining this double structure. We did this with shared state examples that ran for both implementations.
00:13:40.160 This meant we only had to write the test once, but implement both versions.
00:13:52.080 The second example I have is a Rails application we took over from a new client. This application started with 220 gems and a liberal usage of Active Record.
00:14:02.700 The application came with 2,600 tests, of which 730 were controller tests, and the rest included various types of tests, like Capybara and Cucumber tests.
00:14:14.390 The entire test suite for this application ran in around eight and a half minutes.
00:14:28.560 We thought, okay, let's optimize it our way to gain more speed. Running a single controller with the existing application, which required Rails, always took around 14 seconds.
00:14:41.330 After making all the configuration changes, we managed to cut it down to nine seconds, which felt somewhat better, but it wasn't as efficient as before.
00:14:54.540 For a non-controller spec, it looked a little better. Running a single non-controller spec used to take around six and a half seconds.
00:15:05.400 We were able to cut this down to around half a second, providing a better optimization than with controller tests.
00:15:20.210 However, we couldn't just apply this optimization across the entire codebase because manually adding all required statements took a lot of effort.
00:15:33.880 I did all the benchmarking for two controllers, and by the end, I had my good status show me that I had 150 tests to run.
00:15:44.060 Since this was a client project, we couldn't make breaking refactors just for the sake of refactoring.
00:15:56.580 Instead, we needed to find ways to slip these changes into new features as we added them.
00:16:10.000 So, we couldn't simply flip the switch and say, 'Yes, now your application is faster with a better development experience.' However, we projected that we could cut down the runtime from eight and a half minutes to five or six minutes.
00:16:22.210 This was still too long for my taste, but it was an improvement.
00:16:34.090 With all the changes and explicit requires, not many people would agree with my point.
00:16:48.360 The main point I want to make here is that launching a test suite shouldn't be a daunting task. You should be able to run your tests multiple times a minute to get fast feedback.
00:17:04.060 If you take away just one thing from this talk, it's this: split up your spec helpers. Create a spec helper that loads your specs and defines global test covers that you want to use.
00:17:20.150 Have a separate Rails helper that requires the existing spec helpers and also requires the Rails environment. This includes the test speed benefit.
00:17:32.660 If you only test a class that doesn't need anything from Rails, you will experience a substantial speed benefit.
00:17:44.240 Now, let’s talk about what constitutes the 'Rails Way.' What does this actually mean?
00:17:58.000 In Rails doctrine, we can read that we value convention over configuration. We made a conscious decision as a team to favor explicitness over impressiveness.
00:18:12.780 I prefer to read explicit and boring code over magical code that makes assumptions. That doesn't mean I don't like Rails; I just favor explicit code.
00:18:26.760 It's worth mentioning that conventions are heuristics—they are not hard rules. You can break them.
00:18:42.220 If you learn something new, you might realize that conventions you've followed for three years may not apply anymore.
00:18:56.250 It's not a bad thing. The 80/20 rule applies here again; all these conventions in Rails work well for 80% of applications, but the remaining 20% might need something else.
00:19:09.150 That's perfectly fine! This flexibility is one of the strengths of Rails—you just might have to put in some extra work to capitalize on the benefits.
00:19:23.450 Earlier, I mentioned I want to talk about general architecture. I'm not going to bore you with outdated material on what software architecture should look like.
00:19:37.000 There are a couple of more recent architectures out there. First, there's Clean Architecture, which defines some entities at the core of the application, orchestrated by certain use case implementations.
00:19:53.920 These use cases are exposed through controllers in a Rails application, for example, through the web. Similarly, there's a hexagonal architecture, coined by Alistair Cockburn, that follows a similar direction.
00:20:05.780 At the core of your application, you have entities, and you can provide various adapters for different clients. You might have a GUI adapter, or a REST adapter that connects to Salesforce.
00:20:17.560 Last but not least, in Martin Fowler's book, "Patterns of Enterprise Application Architectures," there's the service layer—an idea that again illustrates you have a user interface connecting to a service layer that guards your domain model.
00:20:30.440 The service layer in turn has access to the datasource layer, which represents your database at its core.
00:20:44.800 If we look at these three architectures, they all share a similar idea: it's not something completely new. If we take the service layer as an example and zoom in, it resembles layered architecture.
00:20:56.670 Nothing new to learn; we're back in the past with new words for old concepts—essentially, isolating your application from the outside world.
00:21:09.500 It’s important to remember, though, that Rails is not your application. Business concerns should be paramount.
00:21:23.110 I want to show you an example using an e-commerce platform. Imagine by having a user class; when a user logs in, they represent a customer.
00:21:38.740 This user could have a basket, containing several products that each have categories. As we check out, the user selects a delivery method and a payment option.
00:21:54.260 At the end of the payment process, we get an invoice as a PDF download. This part is the core of our application—there is no controller present, no session hash, or any of the Rails specifics.
00:22:05.150 This aligns with the principles of Domain-Driven Design, a book I highly recommend.
00:22:18.860 To tie this back to the architecture, if we look at how this applies to our Rails app: we start from the top down. The user interface typically is the browser.
00:22:33.450 This browser connects through HTTP to our application, such as our checkout controller, which translates the web request to the checkout use case implementation.
00:22:49.970 This instance and checkout process mediates between several domain model objects, which should ideally just be plain old Ruby objects defining their own behavior.
00:23:01.500 Ultimately, this culminates in creating an order, which serves as our gateway to the database.
00:23:12.590 At its core, it's a straightforward process: a request comes in and interacts with something in the database. How does this link back to the MVC concept?
00:23:26.720 MVC has become more of an abstract concept than a strict design pattern, especially noted in the Rails community where the value of skinny controllers over fat models was discussed.
00:23:38.850 But we realized that these models became hard to test, leading to the opposite extreme of keeping the models thin and overloading controllers with behavior.
00:23:50.690 It isn't sufficient to merely consider these three roles; I like to envision it as a balloon.
00:23:59.880 If you squeeze one end of the balloon, you're not eliminating air or responsibilities; you're redistributing them to the other side.
00:24:13.000 It’s essential to not fixate on one aspect to the exclusion of others. Focus on keeping everything healthy.
00:24:25.770 By dispersing responsibilities into smaller, manageable pieces, we create a codebase that is much easier to understand.
00:24:39.450 If we only concentrate on classes with 5 to 10 lines of code, it becomes inherently easier to follow than wading through a 50-method long monolith.
00:24:53.070 While preparing for this talk, I had a conversation with a friend during coffee about the style we’ve practiced in Rails for years.
00:25:05.190 He asked me if I would do it again, if I would still implement my expectations in this way. I thought for a moment.
00:25:17.270 I agree with all the preloading and loading optimizations, but the concept of Screaming Architecture—removing the app directory and shifting everything?
00:25:30.240 Probably not. I think that idea is overrated. If you consider the default directory structure, you don't need to identify it as a movie organization or an e-commerce platform.
00:25:45.670 It's acceptable to start with something simple—like, 'This is a reservation system.' If I want to know more, I look into lib or source for the namespaced responsibilities.
00:26:01.080 With the separation of concerns, I like the idea of having these physically separated—my core application in lib, with nothing Rails-specific leaking in there.
00:26:19.790 Everything Rails and web-related should remain in app. This approach simplifies the upgrade process, as we wouldn't need to worry about anything in lib.
00:26:34.880 We might wrap some Active Job dependencies, but that’s about it. All of this comes with trade-offs, though.
00:26:46.770 I'm not trying to sell you promised land here—it's not that magically we'll all end up writing more awesome Rails apps.
00:27:00.290 There are trade-offs and considerable ones. For instance, explicit requires aren't commonly seen in typical Rails applications.
00:27:13.950 Carrying back to the city metaphor at the beginning: we put up some big construction sites and cut through several blocks.
00:27:27.370 It's not without thought, but we need to be aware of the cost—it requires consideration.
00:27:40.460 We work in teams and are not islands, so it needs to be a team decision; the team's effectiveness is more crucial than individual idealism.
00:27:54.500 After all, in the end, it’s a technical detail whether a file for a controller lives in this directory or that directory. Who cares?
00:28:06.750 I typically begin every project like this. If I were to start a new project now, I would wait until it becomes painfully slow before I worry about optimizing.
00:28:20.030 For instance, if I set up a Rails application for people to sign up for my birthday party, I would use the free tier of Heroku, scaffold it and deploy it, then call it a day.
00:28:36.890 It all comes down to how much maintenance you expect for this problem and how much benefit you gain from optimizations.
00:28:52.120 We should also remember to not just mindlessly follow traditions passed on to us; we need to question everything, break things, and fix them again.
00:29:10.110 Use the Leaning Tower of Pisa as an example—its attraction draws massive crowds every year.
00:29:26.270 Is that an indicator that we should build every tower like it? Probably not! And finally, know your tools.
00:29:40.780 Understand why you follow certain rules, but also know when to break them. Make conscious decisions.
00:29:54.400 On that note, I'd like to leave some room for questions, and first, I want to say thank you for listening to me.
Edit
Explore all talks recorded at RailsConf 2017
+105