00:00:05.450
All right, thank you Rose, Caitlyn, and everyone at RubyConf AU for organizing this amazing conference. I know a lot of work goes into events like this, and I truly appreciate the opportunity and honor to be here to talk about taming monoliths without microservices.
00:00:21.840
Just a quick show of hands for audience participation: how many folks here work with Rails in their day job? Okay, that’s almost everyone. And how many are here just out of curiosity about Ruby? Great! So, this talk will apply to both large Ruby codebases and large Rails codebases, but it will particularly focus on very large Rails applications. We will delve into that shortly.
00:00:55.200
By the way, can everyone raise their hand if their Rails app takes a second or more to start? Keep your hand up if it takes five seconds or more. Ten seconds or more? Fifteen seconds or more? Thirty seconds or more? Sixty seconds or more? Oh my gosh, buy these folks a beer!
00:01:07.530
Okay, let's get to it. My name is Kelly Sutton. I’ll outline the structure of this talk, which consists of four parts: I’ll quickly introduce myself, share two stories of a team that tries to break down a monolith, discuss a few tactics that you can employ in your day job for breaking up a large Rails codebase, and finally, I’ll wrap it up with some concluding remarks.
00:01:50.650
About me: I’m a software engineer living in San Francisco with my fiancée Emily and our dog Greta. I have been in San Francisco for four years and lived in New York City before that. If you didn’t catch anything I said before, don’t worry—everyone just wants to know about my dog. Greta is quite fashionable; she keeps up with the latest hairstyles and even sported an ombre look recently. She’s a well-traveled pup and even has a kimono that she brought back from Japan. But that’s not why we are here.
00:02:19.800
We are here to discuss Rails projects that truly push the limits.
00:02:22.390
At Gusto, we process over 1 billion USD per month, serving about 1% of small businesses in the United States. We are responsible for 0.1% of the U.S. GDP. Our codebase consists of about one and a half million lines of code, maintained by approximately 80 engineers, predominantly focused on these Rails monoliths.
00:03:02.800
Payroll is incredibly complicated software to work with, as it combines complex elements of time, geography, money, and people. Each aspect, when dealt with individually in software development, can become quite overwhelming.
00:03:14.620
We often need to make trade-offs while building software at Gusto. Our motto is: correctness is more important than performance. It’s acceptable for submitting a payroll or paying employees to take a bit longer, provided that they receive the correct amount. When it comes to payroll, the federal and state governments can be quite unforgiving if accurate payments are not made.
00:03:24.840
One example of where this trade-off is crucial is how we decide to cache information. The performance of the cache can affect the correctness or the freshness of the data.
00:03:34.120
I want to talk about the challenges that arise when codebases become too large. If you’ve read about domain-driven design or have heard talks about large applications, you may find some familiar concepts here. Essentially, we're discussing the need to modularize applications over time. It’s great that Rails has matured to a point where it’s considered 'legacy' software in some companies, which is a good thing—as it shows that there are companies that have successful software rather than being outdated.
00:04:25.800
I will talk about breaking down a monolith, which starts with understanding what I call the 'swamp' or the 'ball of mud.' This refers to an application that takes 60 seconds to start up, where any small change feels monumental and is likely to break something else.
00:04:43.770
In our scenario, each node represents a Rails model file, and we currently have over 700 models in our application. Each color represents a different team involved in this process. After working on this project for a year, we found that we still have several more years to go. Ideally, the colors would cluster closer together, minimizing the presence of 'God objects,' which refer to convoluted parts of the application that become overly dependent.
00:05:02.979
The coupling and complexity in a monolith really starts with the models. If you don’t focus on the models first, working on an application like this can feel quite constraining.
00:05:17.130
I want to clarify that no animals were harmed in the making of this presentation. The swamp refers to the challenges we face in our application, which aims to fulfill various customer needs, including payroll, HR, benefits, and health insurance. When new team members join or someone gets frustrated and says they want to extract a service, it sounds great in theory.
00:05:57.900
In our example, we decided to extract HR services, which include handling names and addresses, personal identification numbers, and more. After creating a new Rails application for HR v2, we would list out the functionalities that the existing HR v1 offers, say about 14 features, and then slowly integrate them back into the main application.
00:06:25.700
However, things complicate midway. It often turns into someone's full-time job to separate what's in HR v2 and what remains in HR v1. As we continue, we still receive bug reports for the old HR system, leading to a frustrating realization: we've created tribal knowledge. Now, anyone trying to look for an HR concept must ask if it’s in HR v2 or if it’s still hanging around in HR v1.
00:07:11.220
One of the better approaches is to introduce a clearer structure. Returning to the HR and payroll example, let's define boundaries. We're using a conceptual boundary here indicated by a separating line between HR and payroll. This boundary serves as a translation layer to clarify the handling of employee information.
00:07:49.960
The process involves taking the Employee Active Record object from HR and converting it into something useful for payroll. While this may seem trivial, it becomes essential as we need to handle various nuances—such as the need to address married names or people with gender transitions—where software design plays a crucial role.
00:08:18.849
As we visualize how to handle these details, we create value objects that help define the responsibilities of HR and payroll more distinctly. This strategy can help in elucidating how communication occurs between these two sections of our application.
00:08:47.740
In simplifying this, one can see that the value objects gradually turn back into Active Record objects on the payroll side. This structure offers clarity to the application by defining how information will travel and be utilized.
00:09:16.710
Moving forward, we can have different developers interact with our system without ongoing confusion and, if needed, shift some portions into separate services later on. The transition is smoother when they can separate these layers without starting from scratch.
00:09:44.360
Back at RailsConf in 2018, DHH discussed the idea of conceptual compression. One of the benefits of Rails is how various concepts can be effectively compressed into a single framework, empowering developers to create functional prototypes swiftly. But as your projects evolve, it’s essential to interpret and adjust the functionalities offered by Rails to accommodate the more intricate needs of larger applications.
00:10:33.950
Before implementing some of these tactics, I want to stress: don't rush to your office next Monday proclaiming that you’re implementing everything outlined here without discussions with your team. Think critically and involve your team in these changes.
00:11:01.930
One key tactic is to avoid circular dependencies as they complicate code changes. In a Rails app, it’s often too easy to fall into this trap, as it’s a default coding practice. For example, while it often seems logical for an Employee to belong to a Company, it’s important to assess whether reversing that relationship is necessary.
00:11:56.520
A visual representation of our dependencies might look overly convoluted, resembling a tangled web. Instead, aim for a neat structure that eliminates cycles. It’s all about simplifying the relationships among your code.
00:12:22.560
Utilize value objects for navigating between the different parts of your application. This goes beyond just using Active Record objects; it’s about identifying and extracting the values you need without coupling them too closely to the underlying models.
00:12:45.210
Another tactic to consider is to avoid callbacks. If you read the Rails documentation closely, you won't find explicit warnings against using callbacks, but they can cause unintended complications as application complexity grows. Instead, think about introducing service objects that oversee both the responsibility of creating a Company and sending emails.
00:13:48.960
Although that may seem excessive for simple tasks, as the process of creating a company expands to involve numerous interactions, managing those relationships with service objects becomes simpler than juggling multiple callbacks.
00:14:24.520
As this process unfolds, remember that introducing more nodes into your implicit dependency graph might initially feel overwhelming but could simplify overall structure. We are often better off managing numerous nodes as opposed to creating cycles.
00:15:27.370
One last reminder is to take your time with these adjustments. These alterations may require dozens or even hundreds of pull requests to fully implement, and it's vital to communicate these changes to your product teams beforehand.
00:16:23.040
Make a robust business case for how it's expected to facilitate a quicker and safer development process. No matter how outdated the existing code may feel, patience and incremental changes will bring better results over time.
00:17:06.890
In closing, I want to share a thought by Kent Beck, an influential figure in software development from the extreme programming and agile software movements. His advice is: 'Make the hard change easy, then make the easy change happen.' When we embark on extracting microservices, the challenge usually lies in implementing structure within an unstructured application.
00:17:45.260
Finally, while structuring a monolith can be challenging, it's vital for creating maintainable applications over time. We are all still working on this journey, and perhaps I will have more to share with you next year. Thank you!
00:18:52.690
Thank you so much, Kelly! And perhaps next year when you return, you could bring Greta along for a little introduction; she would be a fantastic addition to the slides.
