Talks

Summarized using AI

Wrangling Large Rails Codebases

Stephan Hagemann • October 26, 2012 • Earth

In the presentation titled "Wrangling Large Rails Codebases," Stephan Hagemann discusses strategies for managing and organizing large applications within the Rails framework. He emphasizes that building large applications often leads to complexity and chaos, advocating instead for smaller, modular components that can mitigate cognitive overload and enhance maintainability. Key points of the talk include:

  • Morning Stand-ups: Hagemann introduces the practice of morning stand-ups, highlighting the importance of concise communication among team members to prevent redundancy.
  • Avoiding Large Applications: He argues against the necessity of building large applications, insisting they can complicate development and lead to unwieldy codebases.
  • Importance of Structure: Hagemann stresses the need for structured code to enable easier modification and maintenance, comparing disorganized code to clutter in a physical space.
  • Modularity with Gems and Engines: By using gems and engines, developers can encapsulate functionality and maintain clearer dependencies. Hagemann discusses the concepts of higher cohesion and lower coupling, emphasizing how they contribute to a cleaner, more organized application framework.
  • Example of a Simple App: Hagemann describes his app, "The Next Big Thing," ostensibly as a practical reference for illustrating how smaller applications can operate efficiently.
  • Engine Implementation: He explains engines as a means to compartmentalize features and manage migrations without polluting the primary application, thus simplifying the development process.
  • Testing and Communication: The presentation highlights the importance of defining interfaces for inter-communication between engines and emphasizes the role of testing in maintaining code quality.
  • Takeaway on Structure: Hagemann concludes that defining functionalities in terms of gems and engines fosters better organization within Rails applications.

The overarching message encourages developers to think outside traditional Rails paradigms to embrace a modular approach, enhancing efficiency and clarity in large codebases.

Wrangling Large Rails Codebases
Stephan Hagemann • October 26, 2012 • Earth

Help us caption & translate this video!

http://amara.org/v/FGgE/

Rocky Mountain Ruby 2012

00:00:08.650 Good morning, everyone. Let's get this started! I'm about to wrangle some stuff. Trust me, this is the talk, and I'll explain what I mean.
00:00:15.129 The title of my talk is "Wrangling Large Rails Codebases." My name is Stephan Hagemann, and I work for Pivotal Apps, where I pretend to be productive.
00:00:20.439 First, I want to ask: how many of you do a stand-up every morning? We have a practice where every morning we quickly discuss what has been happening, what is planned for the day, and any potential issues. Who's doing that? Have you ever repeated what someone else said?
00:00:35.530 Some of you have, so I like to keep stand-ups simple; everyone has to share something. Then, you can just reiterate what your pair said, which is quite nice. I want to summarize what was discussed in all the talks from yesterday morning.
00:00:57.309 I hope that by the end of this talk, you’ll see that I fit in with those discussions. While they focused on very small topics, I will be discussing much larger concepts. If a talk is like a steak, then this one is definitely not well-done. I hope it’s well-received, but it’s probably more medium rare!
00:01:17.200 The reason is that just two days ago, I had a conversation with my boss, Sandi, at our office. After that, I decided to redo everything, which is why I’m looking for your feedback as we go along. I will be using the next big thing as a reference for two of my examples, so keep that in mind.
00:01:36.430 Just a note: nothing I might say is something that others have stated unless I directly cite them.
00:01:48.130 Now, let’s talk about large applications. In fact, I would argue that you should never build large apps. Does anyone have thoughts on that? I would say it’s hard and can lead to a mess.
00:02:01.900 Sandi once joked that your code may want to kill you, and I had never thought of it that way. I always believed it to be the other way around.
00:02:19.550 Let's explore the wrangling theme a bit more. Just a couple of days ago, I discovered something interesting during a tequila tasting—someone told me I didn’t really know what wrangling meant. I thought it meant tackling something, like in football.
00:02:37.660 As a joke, one of you wanted to bring me Wrangler jeans so I could walk like a cowboy. Sorry, I only have this hat. If anyone is from Texas, you’ll know that it says, "Everything's bigger in Texas," on one side. If what Sandi says is true, then my code is surely out to get me!
00:02:55.590 By the end of this talk, I hope we can all feel more like cowboys or at least have a more relaxed approach to large codebases.
00:03:11.840 Partly because my brain is melting under this hat and because I might look ridiculous, let's move on. Again, I want to emphasize: never build large applications.
00:03:23.330 I built a tiny app called "The Next Big Thing." What is that, you might wonder? Well, it doesn't do much. You can sign up, but there’s ultimately nothing about that new feature.
00:03:35.100 I realize I should have made that smaller, but it doesn’t matter now. All you can do is sign up, and if the server ever wakes up, it will say, "Thank you for signing up." Right now, it looks like everything is still down.
00:03:43.200 So that’s what you can do with the app. Now, back to big apps: who has ever worked in a Rails codebase where the model's directory looked like a complete mess?
00:04:02.160 I feel your pain! I’ve taken the liberty of spreading that out, and it truly is a disaster.
00:04:10.100 I like to think that what I write is a pile of crap because if this is a massive pile, it's a massive pile of crap. I want several things for my application: I want structure.
00:04:25.500 To remind myself, I want to reference Ben, who said that when he had a method with a good name, the implementation was crazily complicated and unmanageable. Why isn't there more structure in a Rails app?
00:04:39.060 Another person I want to mention is Roy—he had a great approach to HTML structure. Everyone seems to be searching for that structure. I want my individual pieces to be understandable and agile.
00:04:56.420 Not in a process way, but rather in the sense that when a small feature request comes up, I want to make small changes without needing to dive into the chaos.
00:05:12.540 Ideally, the code should remain focused and manageable. We should prevent the big pile. We should reduce cognitive overload and allow for enough flexibility to enable change.
00:05:30.120 However, that’s not happening right now. I should say that it might be happening, but not for me at this moment. Wouldn't it be great if I could structure that big pile into smaller, more manageable pieces? Organizing code is like recycling—there's a vast difference between throwing it in a waste bin and putting it in a recycling bin.
00:06:05.700 So, in my analogy of a Rails application, this is a crucial difference that I want to discuss today. What if we could sort our codebase by filename and structure it a little better?
00:06:24.860 Yes, we can! We could use namespaces and modules to organize everything. We can define models clearly. It feels great to retrieve one more layer of organization!
00:06:42.470 But I still struggle with maintaining a smaller focus when everything is still lumped together. Wouldn't it be wonderful if I could isolate these modules into their own encapsulated environments?
00:07:00.300 Let's think about gems. You can create a gem that houses even a small piece of functionality that handles its own specifications. If you design it properly, that gem should live in its own space to prevent it from being cluttered with other codes.
00:07:17.920 Eric Evans wrote a book called "Domain-Driven Design," in which he speaks about modules that convey the story of your system containing a cohesive set of concerns. It’s not just about imposing structure; it’s about discovering the underlying structure that inherently exists.
00:07:35.660 In other words, it’s about finding cohesion without the chaos. To continue with this analogy: when you store away all your winter gear and summer gear together, it becomes a challenge to find anything against the mix of clothing.
00:07:55.180 Higher cohesion yields a simpler view of the world, allowing for looser coupling to the outside. So we’re looking for ways to pull our gems apart and introduce a bit more organization.
00:08:09.020 Let’s analyze modules based on their external interactions. The tighter the cohesion between components, the less outside interaction is needed—similar to organizing your winter clothing apart from your summer clothing.
00:08:29.070 As you can see, we need structure and separation to simplify the complex relationships at play. Now let’s see this app in action, which is our test example.
00:08:50.780 So I’ve made this box, and to illustrate better, I once built an "Annoyance Gem." It has two public methods. Because I don’t want to clutter the screen with too much information, I didn’t show you what they actually do.
00:09:07.860 But essentially, you can query the annoyance level, and it will tell you just how annoyed it is. It can also adjust the text you feed it. This little gem serves as an isolated piece within our application.
00:09:21.420 Every gem comes with its own set of rules and builds in a defined structure, making it much easier to organize.
00:09:37.370 Remember that gems provide explicit dependencies, and with gems, you know that they encapsulate a certain behavior. Just like Eric Evans pointed out, it’s important to identify those cohesive concepts and extract them as gems.
00:09:55.550 For all of the recent projects in our Boulder office, we’ve focused on these gems that are modular and encapsulate specific functionality. It's essential to note that you’re likely already using some form of gem to manage dependencies, such as payment services.
00:10:11.540 The crucial point is that by turning complex functionalities into standalone gems, you can maintain a cleaner application structure. This leads us to a better understanding of the dependencies—and from there, you can easily prove their existence.
00:10:28.650 Now, just to clarify, DHH might pop up at some point and question whether you’ve truly utilized Rails to the fullest or if all you do is work with functional compartments. That's something to think about!
00:10:54.680 Engines are not just for pagination or administration. Rails engines allow you to wrap distinct Rails applications or certain functionalities to share between applications.
00:11:08.125 I believe you should share your engines with yourself! It’s not just about sharing between different apps.
00:11:20.410 Yes, some people find engines to be a bit magical or complex, but let me try to demystify them using a simple example from "The Next Big Thing." This app doesn’t have an app holder as it is all contained within a single engine.
00:11:36.950 This engine, which I call "Teaser," encapsulates the core functionality of the app. Let's explore how engines work and how you can effectively implement them.
00:12:00.230 One way you can achieve functionality between your app and engines is to mount these engines. You can structure engines properly, maintain high cohesion, and let them handle their internal operations independently.
00:12:18.110 Further, for migration management, you can utilize these engines to manage your database migrations without cluttering up the main application database. This is critical for maintaining clarity between your app's core logic and the engines' functions.
00:12:36.520 Assets are another crucial aspect when integrating engines. With engines, all of your images or asset files are confined to their isolated namespaces, reducing confusion in organization.
00:12:51.250 I remember a project where we started with multiple engines. The setup may look complicated, but by labeling everything, we avoided the chaotic environment that often accompanies traditional Rails applications. It simplified our workflow.
00:13:05.350 I presented the "Teaser" app as an underpinning example—a TV-like production hub that integrates different functionalities such as sending text messages while streaming.
00:13:24.520 Engine architecture means that both gems and engines can boil down to encapsulating specific functionalities while maintaining the connectivity to the main app.
00:13:46.200 When implemented correctly, structure will improve by reducing the overall complexity we often associate with traditional Rails apps.
00:14:02.890 Now, transitioning from a traditional Rails application toward a model of using engines can be tough. You know where everything goes within a Rails application, but taking different approaches may leave some developers asking where specific components now reside and how they interact.
00:14:22.530 That said, this shift requires some learning; and if you embrace this transition and put the effort in, it will pay off. Consider how every gem you build will come with its own namespace—think of every Rails application as a world without boundaries, while a gem provides a box for good organization.
00:14:45.980 My takeaway for you today: every gem gives you a namespace that can help you provide structure in your application. In your next Rails project, try to define everything in terms of gems and engines.
00:15:01.870 Let’s start to think outside of the traditional parameters of Rails and adopt this new model. Thank you, everyone!
00:15:14.710 Now, do we have any questions?
00:15:16.210 Yes, of course, let’s have some questions!
00:15:25.360 The question is: what should you do if you find a gem has become too large or problematic? You need to address this carefully.
00:15:43.369 If you keep your migrations within each engine, it can present a challenge to move things as needed, but it’s vital to address it rather than ignoring it.
00:15:56.090 Eric Evans emphasizes doing so is necessary for the health of your application. If you aren’t prepared for such necessary modifications over time, that can lead to larger issues in the future.
00:16:08.700 So while it may be painful today, these pains can relieve a greater potential pain down the road.
00:16:19.870 No, you don’t need to restart your server every time. You can change your engine without having to restart your Rails application, which increases your development efficiency.
00:16:41.170 Engines were a bit more challenging in Rails 2, but in Rails 3 and later, this became much more manageable.
00:16:57.888 If you have engines dependent on each other, it’s crucial to ensure you define strong interfaces between each engine as they can communicate without inappropriate access.
00:17:25.300 As you work on your applications, focus on defining clearer interfaces, so each engine can communicate effectively without causing a ‘Pandora’s Box’ scenario where everything is tightly coupled.
00:17:45.050 Testing becomes vital when you have multiple engines integrated. In CI, a well-structured application can mean fast builds despite numerous dependencies.
00:18:02.270 It’s critical to facilitate testing across these interconnected components, ensuring that all are checked without compromising the efficiency of the build process.
00:18:19.190 Yes, in-engine testing might feel cumbersome, but structured effectively, it is manageable, and your tests will run efficiently.
00:18:30.720 To summarize, you can have clearly defined models, views, and controllers within an engine as long as you've structured your engines properly.
00:18:43.440 Your engine should have tightly coupled functionalities, but they should be independent enough that one does not adversely affect the other.
00:18:57.160 If there are no other questions, I thank you for your attention. Let’s continue to expand on how we can wrangle our large Rails codebases effectively.
Explore all talks recorded at Rocky Mountain Ruby 2012
+12