Birmingham on Rails 2020

Opening Keynote: Structural Engineering in Ruby

Opening Keynote: Structural Engineering in Ruby

by Stephan Hagemann

In his morning keynote at Birmingham on Rails 2020, Stephan Hagemann discusses the importance of structuring Ruby applications using a component-based approach. He uses analogies of towns and cities to emphasize that as applications grow in complexity, they require a corresponding structure to manage this complexity. Hagemann argues that components are essential for taming this complexity, stating that they provide labeled, unique content with explicit dependencies. His presentation is structured in three main parts:

  • Structure Applications with Components: Hagemann explains how components can be used to organize code and manage dependencies more efficiently, helping developers structure Rails applications more effectively. He mentions gems and engines as vehicles for componentization.

  • Challenges within Ruby on Rails: He addresses the limitations of Rails in this respect, acknowledging that Rails does not natively support components in a way that other languages do. Hagemann points out a significant problem: the cost of implementing components in Rails can be high due to the complexity and amount of code generated, particularly for Rails engines.

  • Call to Action: Hagemann encourages developers to participate in sharing their experiences, to help improve the tooling, and to work towards decreasing the overhead of using components in Rails. He suggests ways to tackle issues in managing dependencies across multiple components and emphasizes the importance of community involvement in these improvements.

To illustrate his points, Hagemann provides examples of how components function within an application, examining gem files and routes to demonstrate the structure of a Rails app built with components. He concludes by reiterating the notion that as applications scale, thoughtful architecture and separation of concerns become vital to maintainability and testing efficiency.

Overall, Hagemann's talk is a call to innovate and refine the practices surrounding Rails applications, promoting a component-based architecture as a solution to the challenges posed by complexity in software development.

00:00:16.820 Thanks for having me. I'm one of those speakers from one of those four countries. I'm originally from Germany. This is a small German town. Small towns, whether in Germany or elsewhere, tend to have small roads leading into them. You might not be surprised if this road leads to a town. The biggest German city is Berlin, which is not that big with about three and a half million people.
00:00:33.390 But when you have a bigger city, you would be surprised to find that little road leading into that city. In fact, when you have a bigger city or structure, you create more infrastructure to support it. By our standards, this is not a particularly huge highway, but it's bigger than one for a small town, and you need it to support the complexity of the larger entity. It can get huge. This image shows a highway interchange in China feeding a couple of highways around a small Chinese city of around eight million people.
00:01:09.900 So, it can get very, very big. Similarly, when you're working on these kinds of structures, there’s a parallel to what happens in towns. In a small town, you might have a small side of construction and a little bit of annoyance for a few folks. If you're working on something bigger, it's probably going to annoy a lot more people and take a lot longer. I just heard this morning, though, that in Birmingham, things are processed way faster than everyone thinks, and it's cheaper. So, you all have that figured out.
00:01:25.939 Generally, where I'm from, we don't have this figured out. What I'm trying to say is that the infrastructure you have will adequately support your project only if it's the correct size. To get a sense of what the room is like and to help you wake up properly this morning, let’s do a bit of audience participation. Please stand up if you're able; there are a couple of points I want you to reflect on.
00:02:01.430 So, stand up when you see yourself or your project reflected here: if you have a codebase that has more than 50 models. Okay, there we go. It's not everyone yet. Now, if you have a class in there with more than a thousand lines of code, please stand up. I don’t know why I’m trying to yell while you stand, but I feel like that’s appropriate. A couple more folks stood up. Do you have an Active Record model with more than 30 'has many' relationships? Can you run your entire test suite on your machine in under 20 minutes?
00:02:31.180 Okay, look around you. There are a bunch of you. Hi, good morning to all of you! And everyone else is free to stand up too if you need a stretch. Thank you so much for participating. I would argue that in the analogy of towns and cities, you are probably building something bigger akin to a city rather than a tiny thing that is more akin to a town.
00:03:17.170 Let’s call on the structural engineering in Ruby. This slide is purple for two reasons: one, because I've been accused of burying the lead and not starting with the most important point; two, because when Brian and I discussed this talk, he mentioned how I had done a workshop, and he suggested I could talk about components or component architectures. I took that very seriously, and I thought about what else supports better architecture.
00:03:50.780 Here’s my bold statement: the only thing that can ultimately help you reign in the complexity of your application is components. The caveat is that this applies within one app. We will hear from Yulia later about how we want apps to communicate, which is also a very important topic. However, the fundamental building block of applications is components. This is an additional bold statement: the Ruby language doesn’t have native components. However, gems can act as components and work well for Rails, although not ideally, and we will delve into why.
00:04:36.250 So, I have three parts to this talk this morning. First, I will give you an overview of how I structure applications using components. Next, I will explain why I made my last statement and why it doesn’t always feel right with Rails. Finally, I want to call you to action and get your help in making things better.
00:05:53.020 This discussion is based on a book I published a couple of years ago called "Component-Based Rails." After this talk, you won’t need to read it because I am giving you the short version. Let’s go over a very small application that is organized using this component structure. First, I want to point out that at the top of this structure, there is no 'app' folder.
00:06:11.830 So, if there’s no 'app' folder, where’s the functionality? We can still have functionality; we pull in various sorts of functionality via our dependencies. So let's look at the Gemfile, or rather a part of it. In this specific Gemfile, you will see a construct that adds a path for components, which loads several gems. This tells Bundler to look locally into the folder structure for these dependencies.
00:06:36.550 If we navigate over to that folder, we will find those gems listed as well as a few others. But if we look down the routes file, we can see if the application actually delivers something. Sure enough, it does! You may notice those are exactly the same gems mentioned earlier in the Gemfile that are being mounted here. The functionality of this application comes exclusively from components.
00:07:12.049 I will discuss three of those components to explain some of the things you might see. Let's look at games first. The first thing I like to do when examining a gem is to look at the gems it depends on, as this gives me an idea of how things are functioning. The first noticeable aspect is that we depend on Rails, though not on all of Rails - just Active Record. I realize you might have difficulty with the font size if you’re seated toward the back, so I apologize for that.
00:08:14.750 We are dependent on only some parts of Rails, but we also rely on another gem called 'teams.' For those who paid close attention when the list of gems scrolled past, you will notice this one of those local gems that hasn't yet been referenced. If that’s the case, let's open the Gemfile, and indeed, we see that path construct once again — this time with a '../' because we're referencing the same components folder.
00:09:12.040 This gem depends on other gems inside the same application structure. You can start to see that these dependencies span the complexity of this application. Let's keep going. The entry point for a gem in Ruby is typically the file named after the gem in the 'lib' folder. So, if we look into that file, there isn’t much happening; we just load another file, namely an 'engine' for this game gem.
00:09:32.560 Opening up that engine reveals some interesting details. We’re using a Rails engine to make this component work. There's an 'isolated namespace' statement, meaning that all Rails tooling expects and creates everything inside this 'games' namespace, ensuring that our gem can act independently without colliding with functionality from other parts of the app.
00:10:44.030 This leads us to one of the pitfalls we encountered when we were initially building out applications with this technique, which is a little hack that ensures that any migrations in this gem get automatically picked up by any gem that includes this component. We also observe that there are no Rails dependencies except for Active Record, so no routes are specified. Now, let’s look at another component, 'games admin,' which was one of the gems listed in the original Gemfile of this application.
00:11:51.860 The gem specification here does depend on Rails and a bunch of other things. If we check the routes this time, sure enough, there is a resource for managing games, and behind this, it's just a scaffold for administrating games.
00:12:17.940 Recall, we looked at the routes file earlier that referenced this specific route file and mounted the routes from this engine at that location. Let’s look at one last component, which is called 'predictor.' Again, we see that there is no dependency on anything Rails, but we do have a dependency on a gem that calculates the relative strength of competitors in games.
00:13:00.130 When we inspect the file of the gem’s name in 'lib,' it looks like a normal gem. We're loading that external dependency, as well as some internal dependencies. Additionally, when we review the models, we can see that we have tests for that model. Remember, we are still operating inside an engine that is part of an application which stands on its own.
00:13:34.050 I want to highlight this test script at the bottom. It is essentially a shell script that does everything necessary to run the tests for this specific engine. The reason behind this approach is that different engines might need different things to be tested, and/or they should be set up differently.
00:14:12.730 By creating this little script, we can standardize. Finally, there was no Rails dependency and only Active Record, so unsurprisingly, there are no routes here. Now let's examine another component where we will see that change. When we assess 'games admin,' one of the gems listed in the application's Gemfile, we see that it does depend on Rails and several other things.
00:14:53.600 We notice we have controllers and views. Checking the routes again confirms there is a resource for managing games, and that is based on a scaffold for administrating games. We previously explored the routes file referencing this route file and mounting the routes from this engine.
00:16:06.090 Let’s look at one last of these components. This one is called 'predictor.' Again, it does not depend on Rails; but this time, we see a dependency on a gem used to calculate the relative strengths of competitors in games. However, a similar observation remains, where we may have an Active Record model inside this gem.
00:16:57.030 When examining the file for the gem in 'lib,' we find it looks like a standard gem. Here, we encounter the loading of an external dependency alongside some internal dependencies. Upon further inspection, we can see one test script for the engine, which is simpler this time, requiring hardly any setup.
00:17:40.700 We merely run the bundle in this case and run the tests for this component. In summary, the test scripts we've looked into all have similar functions in the context of creating a build, where we find all of the test files, run the tests, and only if they all pass will it yield a green build, otherwise, you will see a red build.
00:18:57.260 Thus, we have split our test suite into various components, yet we still can compile all results to report on the application status. The point is that this is just Rails with a little bit of structure, a new strategy that can help us manage complexity.
00:19:13.840 As we lay out this structure, I want to remind everyone about what that means for our techniques and structure. It’s essential to visualize it appropriately and to think through the way we structure our applications better. I mentioned earlier there is an urge to not get lost in methods.
00:20:21.330 Methods enable code reuse, allowing us to call the same logic multiple places without repeating ourselves. Methods encapsulate logic and render the process cleaner, hiding complex processes behind intuitive function names. Each method's cost boils down to simply defining it, comprising two lines of code: the definition line and the end line.
00:21:25.680 Similarly, with classes, creating one bundles methods, which adds data through objects we create based on those classes. This mechanism, again, is low cost with just a class name and end line. Regarding modules, we gain benefits related to reusability and help prevent naming collisions, ultimately clarifying relationships.
00:22:19.180 Ultimately, components can alleviate complexity, and therefore it begs the question about their unique aspects. The way I define components includes two distinct characteristics: they must be labeled and contain unique content, and also have explicit, directed, acyclic dependencies.
00:23:07.230 To dive deeper, we need to refine our definition of components. Gems can be seen as labeled content with unique purposes. They tend to maintain explicit dependency structures. While many label themselves in a universal manner, the challenge can often be in distinguishing differences, like trying to identify various brands of similar products at the supermarket.
00:24:29.540 Moreover, consider that the organization of components allows for better shared properties, in organizational content and consumer discernment. The consumer should seek clarity in how dependencies function and interrelate with one another, thus supporting a collaborative approach. Through this visual representation, we can facilitate clearer communication among development teams.
00:25:50.690 Even more, we recognize that creating a gem is straightforward in terms of dependencies, but we must consider them correctly, contextually. Bundler can efficiently manage this complexity, but gems aren’t limited to that function without concrete organization to enforce clarity and manageable dependencies.
00:27:20.800 By performing cost-benefit analysis across our components, we can understand that each component's testing adds valuable independent testing layers throughout the overall application. Working with structured layers helps delineate functions to scale complexity, effectively focusing development efforts on core features beyond performance benchmarks.
00:28:47.240 Finally, I urge us to consider tools that enhance productivity in tandem with this architecture framework. We must seek to reduce coding overhead that contributes unnecessary complexity. The key challenge will remain how we approach those tools and practices within the Ruby ecosystem.
00:30:27.160 While the idea of microservices has gained popularity, we must carefully reflect on how to trickle components into existing architectures; nonetheless, we should always return to solid verifiable structures.
00:31:40.110 At this point, reflective thought for developers requires a distillation of ideas: working together to face architectural hurdles, each development philosophy can pave ways towards stronger solutions.
00:32:35.950 Continuing to cultivate meaningful discussion and resource sharing at events empowers our community to tackle modern challenges collectively. So I encourage each of you to engage with events, share insights, and contribute to that broader dialogue.