Talks
Wrangling Large Rails Codebases
Summarized using AI
Wrangling Large Rails Codebases
by Stephan HagemannIn the presentation titled "Wrangling Large Rails Codebases," Stephan Hagemann discusses strategies for managing and organizing large applications within the Rails framework. He emphasizes that building large applications often leads to complexity and chaos, advocating instead for smaller, modular components that can mitigate cognitive overload and enhance maintainability. Key points of the talk include:
- Morning Stand-ups: Hagemann introduces the practice of morning stand-ups, highlighting the importance of concise communication among team members to prevent redundancy.
- Avoiding Large Applications: He argues against the necessity of building large applications, insisting they can complicate development and lead to unwieldy codebases.
- Importance of Structure: Hagemann stresses the need for structured code to enable easier modification and maintenance, comparing disorganized code to clutter in a physical space.
- Modularity with Gems and Engines: By using gems and engines, developers can encapsulate functionality and maintain clearer dependencies. Hagemann discusses the concepts of higher cohesion and lower coupling, emphasizing how they contribute to a cleaner, more organized application framework.
- Example of a Simple App: Hagemann describes his app, "The Next Big Thing," ostensibly as a practical reference for illustrating how smaller applications can operate efficiently.
- Engine Implementation: He explains engines as a means to compartmentalize features and manage migrations without polluting the primary application, thus simplifying the development process.
- Testing and Communication: The presentation highlights the importance of defining interfaces for inter-communication between engines and emphasizes the role of testing in maintaining code quality.
- Takeaway on Structure: Hagemann concludes that defining functionalities in terms of gems and engines fosters better organization within Rails applications.
The overarching message encourages developers to think outside traditional Rails paradigms to embrace a modular approach, enhancing efficiency and clarity in large codebases.
00:00:08.650
Good morning, everyone. Let's get this started! I'm about to wrangle some stuff. Trust me, this is the talk, and I'll explain what I mean.
00:00:15.129
The title of my talk is "Wrangling Large Rails Codebases." My name is Stephan Hagemann, and I work for Pivotal Apps, where I pretend to be productive.
00:00:20.439
First, I want to ask: how many of you do a stand-up every morning? We have a practice where every morning we quickly discuss what has been happening, what is planned for the day, and any potential issues. Who's doing that? Have you ever repeated what someone else said?
00:00:35.530
Some of you have, so I like to keep stand-ups simple; everyone has to share something. Then, you can just reiterate what your pair said, which is quite nice. I want to summarize what was discussed in all the talks from yesterday morning.
00:00:57.309
I hope that by the end of this talk, you’ll see that I fit in with those discussions. While they focused on very small topics, I will be discussing much larger concepts. If a talk is like a steak, then this one is definitely not well-done. I hope it’s well-received, but it’s probably more medium rare!
00:01:17.200
The reason is that just two days ago, I had a conversation with my boss, Sandi, at our office. After that, I decided to redo everything, which is why I’m looking for your feedback as we go along. I will be using the next big thing as a reference for two of my examples, so keep that in mind.
00:01:36.430
Just a note: nothing I might say is something that others have stated unless I directly cite them.
00:01:48.130
Now, let’s talk about large applications. In fact, I would argue that you should never build large apps. Does anyone have thoughts on that? I would say it’s hard and can lead to a mess.
00:02:01.900
Sandi once joked that your code may want to kill you, and I had never thought of it that way. I always believed it to be the other way around.
00:02:19.550
Let's explore the wrangling theme a bit more. Just a couple of days ago, I discovered something interesting during a tequila tasting—someone told me I didn’t really know what wrangling meant. I thought it meant tackling something, like in football.
00:02:37.660
As a joke, one of you wanted to bring me Wrangler jeans so I could walk like a cowboy. Sorry, I only have this hat. If anyone is from Texas, you’ll know that it says, "Everything's bigger in Texas," on one side. If what Sandi says is true, then my code is surely out to get me!
00:02:55.590
By the end of this talk, I hope we can all feel more like cowboys or at least have a more relaxed approach to large codebases.
00:03:11.840
Partly because my brain is melting under this hat and because I might look ridiculous, let's move on. Again, I want to emphasize: never build large applications.
00:03:23.330
I built a tiny app called "The Next Big Thing." What is that, you might wonder? Well, it doesn't do much. You can sign up, but there’s ultimately nothing about that new feature.
00:03:35.100
I realize I should have made that smaller, but it doesn’t matter now. All you can do is sign up, and if the server ever wakes up, it will say, "Thank you for signing up." Right now, it looks like everything is still down.
00:03:43.200
So that’s what you can do with the app. Now, back to big apps: who has ever worked in a Rails codebase where the model's directory looked like a complete mess?
00:04:02.160
I feel your pain! I’ve taken the liberty of spreading that out, and it truly is a disaster.
00:04:10.100
I like to think that what I write is a pile of crap because if this is a massive pile, it's a massive pile of crap. I want several things for my application: I want structure.
00:04:25.500
To remind myself, I want to reference Ben, who said that when he had a method with a good name, the implementation was crazily complicated and unmanageable. Why isn't there more structure in a Rails app?
00:04:39.060
Another person I want to mention is Roy—he had a great approach to HTML structure. Everyone seems to be searching for that structure. I want my individual pieces to be understandable and agile.
00:04:56.420
Not in a process way, but rather in the sense that when a small feature request comes up, I want to make small changes without needing to dive into the chaos.
00:05:12.540
Ideally, the code should remain focused and manageable. We should prevent the big pile. We should reduce cognitive overload and allow for enough flexibility to enable change.
00:05:30.120
However, that’s not happening right now. I should say that it might be happening, but not for me at this moment. Wouldn't it be great if I could structure that big pile into smaller, more manageable pieces? Organizing code is like recycling—there's a vast difference between throwing it in a waste bin and putting it in a recycling bin.
00:06:05.700
So, in my analogy of a Rails application, this is a crucial difference that I want to discuss today. What if we could sort our codebase by filename and structure it a little better?
00:06:24.860
Yes, we can! We could use namespaces and modules to organize everything. We can define models clearly. It feels great to retrieve one more layer of organization!
00:06:42.470
But I still struggle with maintaining a smaller focus when everything is still lumped together. Wouldn't it be wonderful if I could isolate these modules into their own encapsulated environments?
00:07:00.300
Let's think about gems. You can create a gem that houses even a small piece of functionality that handles its own specifications. If you design it properly, that gem should live in its own space to prevent it from being cluttered with other codes.
00:07:17.920
Eric Evans wrote a book called "Domain-Driven Design," in which he speaks about modules that convey the story of your system containing a cohesive set of concerns. It’s not just about imposing structure; it’s about discovering the underlying structure that inherently exists.
00:07:35.660
In other words, it’s about finding cohesion without the chaos. To continue with this analogy: when you store away all your winter gear and summer gear together, it becomes a challenge to find anything against the mix of clothing.
00:07:55.180
Higher cohesion yields a simpler view of the world, allowing for looser coupling to the outside. So we’re looking for ways to pull our gems apart and introduce a bit more organization.
00:08:09.020
Let’s analyze modules based on their external interactions. The tighter the cohesion between components, the less outside interaction is needed—similar to organizing your winter clothing apart from your summer clothing.
00:08:29.070
As you can see, we need structure and separation to simplify the complex relationships at play. Now let’s see this app in action, which is our test example.
00:08:50.780
So I’ve made this box, and to illustrate better, I once built an "Annoyance Gem." It has two public methods. Because I don’t want to clutter the screen with too much information, I didn’t show you what they actually do.
00:09:07.860
But essentially, you can query the annoyance level, and it will tell you just how annoyed it is. It can also adjust the text you feed it. This little gem serves as an isolated piece within our application.
00:09:21.420
Every gem comes with its own set of rules and builds in a defined structure, making it much easier to organize.
00:09:37.370
Remember that gems provide explicit dependencies, and with gems, you know that they encapsulate a certain behavior. Just like Eric Evans pointed out, it’s important to identify those cohesive concepts and extract them as gems.
00:09:55.550
For all of the recent projects in our Boulder office, we’ve focused on these gems that are modular and encapsulate specific functionality. It's essential to note that you’re likely already using some form of gem to manage dependencies, such as payment services.
00:10:11.540
The crucial point is that by turning complex functionalities into standalone gems, you can maintain a cleaner application structure. This leads us to a better understanding of the dependencies—and from there, you can easily prove their existence.
00:10:28.650
Now, just to clarify, DHH might pop up at some point and question whether you’ve truly utilized Rails to the fullest or if all you do is work with functional compartments. That's something to think about!
00:10:54.680
Engines are not just for pagination or administration. Rails engines allow you to wrap distinct Rails applications or certain functionalities to share between applications.
00:11:08.125
I believe you should share your engines with yourself! It’s not just about sharing between different apps.
00:11:20.410
Yes, some people find engines to be a bit magical or complex, but let me try to demystify them using a simple example from "The Next Big Thing." This app doesn’t have an app holder as it is all contained within a single engine.
00:11:36.950
This engine, which I call "Teaser," encapsulates the core functionality of the app. Let's explore how engines work and how you can effectively implement them.
00:12:00.230
One way you can achieve functionality between your app and engines is to mount these engines. You can structure engines properly, maintain high cohesion, and let them handle their internal operations independently.
00:12:18.110
Further, for migration management, you can utilize these engines to manage your database migrations without cluttering up the main application database. This is critical for maintaining clarity between your app's core logic and the engines' functions.
00:12:36.520
Assets are another crucial aspect when integrating engines. With engines, all of your images or asset files are confined to their isolated namespaces, reducing confusion in organization.
00:12:51.250
I remember a project where we started with multiple engines. The setup may look complicated, but by labeling everything, we avoided the chaotic environment that often accompanies traditional Rails applications. It simplified our workflow.
00:13:05.350
I presented the "Teaser" app as an underpinning example—a TV-like production hub that integrates different functionalities such as sending text messages while streaming.
00:13:24.520
Engine architecture means that both gems and engines can boil down to encapsulating specific functionalities while maintaining the connectivity to the main app.
00:13:46.200
When implemented correctly, structure will improve by reducing the overall complexity we often associate with traditional Rails apps.
00:14:02.890
Now, transitioning from a traditional Rails application toward a model of using engines can be tough. You know where everything goes within a Rails application, but taking different approaches may leave some developers asking where specific components now reside and how they interact.
00:14:22.530
That said, this shift requires some learning; and if you embrace this transition and put the effort in, it will pay off. Consider how every gem you build will come with its own namespace—think of every Rails application as a world without boundaries, while a gem provides a box for good organization.
00:14:45.980
My takeaway for you today: every gem gives you a namespace that can help you provide structure in your application. In your next Rails project, try to define everything in terms of gems and engines.
00:15:01.870
Let’s start to think outside of the traditional parameters of Rails and adopt this new model. Thank you, everyone!
00:15:14.710
Now, do we have any questions?
00:15:16.210
Yes, of course, let’s have some questions!
00:15:25.360
The question is: what should you do if you find a gem has become too large or problematic? You need to address this carefully.
00:15:43.369
If you keep your migrations within each engine, it can present a challenge to move things as needed, but it’s vital to address it rather than ignoring it.
00:15:56.090
Eric Evans emphasizes doing so is necessary for the health of your application. If you aren’t prepared for such necessary modifications over time, that can lead to larger issues in the future.
00:16:08.700
So while it may be painful today, these pains can relieve a greater potential pain down the road.
00:16:19.870
No, you don’t need to restart your server every time. You can change your engine without having to restart your Rails application, which increases your development efficiency.
00:16:41.170
Engines were a bit more challenging in Rails 2, but in Rails 3 and later, this became much more manageable.
00:16:57.888
If you have engines dependent on each other, it’s crucial to ensure you define strong interfaces between each engine as they can communicate without inappropriate access.
00:17:25.300
As you work on your applications, focus on defining clearer interfaces, so each engine can communicate effectively without causing a ‘Pandora’s Box’ scenario where everything is tightly coupled.
00:17:45.050
Testing becomes vital when you have multiple engines integrated. In CI, a well-structured application can mean fast builds despite numerous dependencies.
00:18:02.270
It’s critical to facilitate testing across these interconnected components, ensuring that all are checked without compromising the efficiency of the build process.
00:18:19.190
Yes, in-engine testing might feel cumbersome, but structured effectively, it is manageable, and your tests will run efficiently.
00:18:30.720
To summarize, you can have clearly defined models, views, and controllers within an engine as long as you've structured your engines properly.
00:18:43.440
Your engine should have tightly coupled functionalities, but they should be independent enough that one does not adversely affect the other.
00:18:57.160
If there are no other questions, I thank you for your attention. Let’s continue to expand on how we can wrangle our large Rails codebases effectively.
Explore all talks recorded at Rocky Mountain Ruby 2012
+12