Bringing Your Rails Monolith Along As The Business Grows

Talks

Carrick Rogers

#software-architecture

#team-collaboration

Bringing Your Rails Monolith Along As The Business Grows

by Carrick Rogers

The video presentation "Bringing Your Rails Monolith Along As The Business Grows" by Carrick Rogers, Director of Engineering at Ontra, discusses the journey and experiences of managing a Ruby on Rails monolith as the company expands. Ontra specializes in processing and analyzing private market legal contracts and has seen substantial growth since its inception in 2016.

Key points covered in the presentation include:

In conclusion, Rogers inspires confidence in the monolithic architecture's viability, emphasizing that systematic planning, robust testing, and community alignment are essential to thriving amid growth and technical challenges. By reflecting on their experiences over the years, Ontra is poised for continued success while nurturing a collaborative and efficient engineering culture.

00:00:12.120 Hello, everyone. I think we have 1:30 on the clock so I should get this show underway. Thanks for joining us today for "Bringing Your Rails Monolith Along As The Business Grows." While I'm up here talking, I'm Carrick Rogers, the Director of Engineering for our Web Applications team. This is very much a group presentation that is made possible by all the hard work of the engineers who have contributed to various portions of our code base. They're down in the front row, and unbeknownst to them, they may have to help answer some questions, especially in areas that I'm not as familiar with.

00:00:51.780 So, to start with, who is Ontra? What do we do? We process, digitize, and analyze private market legal contracts. The simplest way to think of this is that investment banks like to sell a lot of things to each other. They have legal obligations that stem from these transactions, and they need to understand what they have to report to their investors, what they have to report to the SEC, and what needs to be reported six months from now. We also work with unions and pensions, so we have a lot of AI-enabled software that assists specialist lawyers in processing these contracts and pulling out the key terms required for their reports.

00:01:22.200 We are a Ruby on Rails monolith, which might not surprise you, with an Ember.js frontend. We have been around since 2016, when the Ruby application launched. I believe we soft-launched in late 2015. Our CTO wanted to claim responsibility for building the initial round of the application, so he pulled a bunch of all-nighters before our principal engineer, Jerry, joined to avoid having to share the initial credit. We've been using the same code base in a monorepo since inception and currently have about a hundred people who have contributed to it.

00:02:13.439 Now, why this talk? One thing we found, especially as we've been going out to hire and talk to potential vendors and contractors, is that we get variations on the question: "What is your code base like? Why haven't you moved off a monolith or monorepo?" Microservices are very much a trend now, and this leads to the big question: "Are you still on the monolith because you want to be, or because you have to be?" This is often asked in the vein of whether your monolith is just too hard to break down and escape from, leaving you stuck with it, or if you genuinely prefer it.

00:03:10.620 We have thought deeply about this, and we are actually quite happy with our current setup. This is true in terms of our ability to function at scale, and it's also true in terms of having a code base that has many engineers committing into one repository without stepping on each other’s toes. We've been asked this question so frequently that it has gotten us thinking: how did we end up feeling this way? We can think back to certain decisions we've made over the years that led to specific high points or low points with regard to our happiness. Some things paid off while others did not, but at the same time, we realized we had never really undergone a full retrospective to assess how we managed to make it all work.

00:03:52.680 When we tell people that we are happy with our monolith, it's not just a line we feed them as part of a pitch process; we genuinely feel this way. In this talk, if we had a time machine, we would travel back and advise ourselves four or five years ago on what to continue doing and what to reconsider. Hopefully, for folks out there contemplating their monolith's survival and functionality as they grow, they can derive some value from what we share.

00:04:58.800 To illustrate how big of a monolith we are, we're now over 4,500 commits, and it’s safe to say that there's been some activity since I pulled these slides together a couple of weeks ago. We have about 48 different code committers and approximately 62,000 lines of Ruby code across various directories. There are around 112 lines in our spec directory, emphasizing the importance we place on tests. On the Ember side, we have just over 61,000 lines of JavaScript and a little over 33,000 lines of Handlebars. We utilize all the usual suspects in our Ruby on Rails tooling, such as Active Record, Redis, and Sidekiq.

00:05:54.419 We have also developed project Olive Scorpius, a non-existent project, but it showcases our demo environment. This proof of concept confirms that we are a legitimate app, capable of showcasing features like document processing and managing multiple file versions, which we will touch on later.

00:06:11.460 The first thing I want to go over as we look back at our decisions is what we did right and what we may have done poorly. Having those 110,000 lines of Ruby testing in our spec directory has proved to be very useful. When we started out, there were three engineers, and this number eventually grew to five. At that point, our primary focus was generating revenue and proving product-market fit. We were not overly concerned about scaling; we were more worried about simply existing in the year ahead. A key realization was that we managed to create decent testing patterns amidst our growth.

00:07:33.360 In retrospect, we were quite pragmatic about our approach, albeit unintentionally. As I reviewed our Git history and spoke with some of the veterans in our team, I discovered that we initially only had our spec tests for the Rails API. We wanted to ensure the correct JSON response when interacting with the API, inferring that the frontend would handle this correctly and that data would end up in the database accurately.

00:07:54.780 This approach allowed us to move quickly and maintain stability. Later on, we introduced copybara tests and eventually integrated Ember CLI for more comprehensive testing. We now rely on our spec tests for multiple feature tests, which check the proper rendering of elements on the page. I have a code snippet here, which is credited to Ryan Biggs’ blog post. Whenever working with Ajax, we found that copybara tests did not play well with our setup, leading to some challenges. So, we developed a counter system to ensure the DOM was in a ready state, effectively handling loading waits, though we're aware this presents long-term implications as we scale.

00:09:16.140 Today, as we approach six years of existence, we are forming an automated testing team. While we still want our engineers involved in writing automated tests, we’ve reached a size where we can have dedicated automated testing engineers and managers. Overall, we have achieved a 97% coverage rate when leveraging tools like SimpleCov, which checks for any missing lines. This success stems from our pragmatic roots and the gradual addition of resources that allowed for proper testing implementations.

00:10:39.300 It's equally important to highlight some other tools we’ve brought into our workflow over the years. We did not introduce a Leatherman Multi-Tool into our code base, but it’s worth giving a shout-out to Leatherman for equipping our booth with the necessary tools. In terms of our Ruby on Rails stack, we started simply but have since integrated tools such as Brakeman for vulnerability scanning, bundle audit for gem updates, and Rubocop for code style enforcement.

00:11:30.240 I would emphasize that our use of Rubocop is an excellent example of how we began small with its implementation. Over the years, we evolved our usage to write our own Rubocop rules. This development only came a few years in, but the importance of fostering developer ergonomics extended beyond just providing comfortable hardware.

00:11:58.680 Our CTO has been a strong advocate for tools that support testing, linting, and vulnerability scanning. This approach inadvertently helped us adhere to a lot of community practices. By adopting these tools, we were able to address issues highlighted by Brakeman and bundle audit promptly; fixing vulnerabilities and maintaining best practices became a seamless part of our development process. Rather than trying to invent our own patterns for the monolith, we have aligned ourselves with established community standards, which has ultimately paid off by limiting the emergence of new patterns.

00:12:56.640 As we continue to refine our code base, we recognize the necessity of meeting business needs. We observed substantial growth in user activity, which introduced standard scaling problems, particularly those pesky n+1 issues if not caught early. However, the complexity of our user base has increased. We are allowing lawyers to process documents for companies, which now include not just the lawyer needing their document, but also compliance departments that wish to consolidate reports on obligations.

00:14:00.060 Initially, we thought our target audience would mainly be independent lawyers, like a woman residing in Cancun who could work 20 hours a week, earning enough to support her lifestyle. The reality revealed that many lawyers were establishing full-fledged law firms and found value in our services. One such group's legal staffing team began using our platform extensively, so we had to extend our business logic significantly to accommodate this growth.

00:15:08.640 Moreover, we've experienced complexities with our internal customers. In early stages, queries were often directed toward engineers who would manually generate reports, usually in CSV format. Now, we face new customer features, such as integrating APIs, as well as maintaining relationships with various stakeholders. As we navigated these challenges, I've identified several case studies which exemplify our evolving approach to managing our codebase.

00:15:52.440 Early on, we used Active Model Serializers within Ruby on Rails, ensuring smooth transfers of data right to Ember. However, we began encountering scaling and performance issues. To tackle these, we turned to the JSON API Serializer, which was once supported by Netflix as Fast JSON API. It’s important to note that while our Vice President expressed interest in contributing to this code base, he found he’d never worked on it, which dashed some hopes. Through a process of unthinking evolution, we established that we now have 117 Active Model Serializers and 106 JSON API Serializers. When performance problems arose and customers complained about slow page loads, we set out to replace the slow serializers.

00:16:49.740 We created some firm rules: first, address the slow serializers; second, ensure any new development uses the JSON API serializer. This strategic process has led to a manageable migration, and we estimate that about 50 percent is now converted to using JSON API. Having grown accustomed to this project, we can confidently assess that the conversion has not just solved a problem but also preserved our relationship with product development teams.

00:17:49.680 Another topic more specific to our monolithic codebase is related to our initial way of handling enums. Initially, we relied on the Simple Enum gem, which was quite common among the Ruby community. When Active Record Enums were introduced in Rails 4.1, we seamlessly transitioned to using this feature without encountering issues. However, as our data science team grew, they started raising inquiries about certain internal enumerations becoming rather obscure.

00:18:39.060 This prompted us to build an internal `ICCEnum` gem. The conversion was successful, yet it highlighted a recurrent pattern where we have adhered closely to the Rails community’s conventions. While we still have about 90 Simple Enums scattered throughout our code, we previously eliminated unwanted Active Record Enums with data science input, thus refining our practices.

00:19:06.420 Overall, we believe that limiting ourselves to community patterns has capped the maximum number of patterns we can face. It has allowed us to maintain a cleaner codebase that remains comprehensible to everyone who joins, thus improving the onboarding process. With existing documentation supporting earlier Rails versions, new hires can access useful resources pertaining to legacy code.

00:19:59.160 As we come to a close, I want to express that while our RSpec tests served us valiantly initially, we should have embraced Ember tests sooner. Relying solely on RSpec for every scenario slowed our testing suite and resulted in unnecessary overhead. As previously mentioned, relying on a single hammer for every nail proved inadequate, causing mandatory wait times for Ajax processes. Therefore, pushing for Ember tests would have significantly enhanced our speed and efficiency.

00:21:11.220 Thank you all for your attention! I'm happy to answer any questions you may have. I invite you to proceed with your queries.

00:21:36.720 One audience member mentioned, "I find one of the hardest parts of maintaining a monolith is the upgrade process, which you never touched on, and I'd like to know about your upgrade process and if you would have done something differently." My response is that we maintain a proactive approach, updating immediately when a new version of Rails is released.

00:22:12.300 We assign one of our teams to manage the upgrade process. For instance, when I joined during the pandemic peak, we were backlogged on updates, but my principal and I dedicated two weeks over the Christmas and New Year’s break to get everything up to date. We've since maintained this aggressive strategy, ensuring we don’t fall behind—tagging our entire team in these upgrades to encourage prompt reporting of any issues.

00:23:09.960 Thank you once again, everyone. You’ve been a wonderful audience! Please feel free to reach out if you have more queries or insights to share.

RailsConf 2022