Ruby Video
Anthony Eden
10 Years In - The Realities of Running a Rails App Long Term
Anthony Eden
@aeden
#ruby-on-rails
#legacy-applications
#dependency-management
#software-design-patterns
#testing
#team-dynamics
Summarized using AI

10 Years In - The Realities of Running a Rails App Long Term

by Anthony Eden

In the talk titled '10 Years In' delivered by Anthony Eden at RailsConf 2021, the speaker discusses the journey of maintaining and evolving the DNSimple application, a Ruby on Rails-based platform, over a decade. Started in April 2010 with Ruby on Rails version 2.3, DNSimple has undergone numerous changes, reflecting the evolving landscape of software development and the Ruby on Rails framework itself.

Key Points Discussed:

  • History of DNSimple: DNSimple was created as a collaborative effort between two developers, focusing initially on providing DNS automation services. The first version was completed within three months.
  • Software Design Evolution: The speaker reflects on the initial design choices, discussing the 'thin controller, fat model' principle, and how this led to complex business logic being buried in controllers—indicating some of the pitfalls at the inception of the application.
  • Testing Practices: Originally relying heavily on Cucumber and RSpec, the testing strategy evolved due to maintenance challenges. Over time, a shift was made towards using RSpec predominantly for both unit and integration tests.
  • Operational Infrastructure Changes: DNSimple transitioned from using PostgreSQL and MySQL to enhanced architectures such as an anycast system, enabling better performance and scalability. The choice of background job processing also changed from Rescue to Sidekiq, encapsulating dependencies and enhancing reliability.
  • Dependency Management: Early on, dependency management was ad-hoc; however, as the team grew, a systematic approach was adopted with tools like Dependabot facilitating automated dependency updates, significantly improving maintenance efficiency.
  • Team Dynamics and Changes: The speaker emphasizes the impact of team growth on code quality and consistency. The introduction of new developers brought fresh ideas and a need for internal style guides to maintain code uniformity as the team changed over time, balancing diverse coding styles and opinions.
  • Future Challenges: Looking ahead, the ongoing challenge includes managing complexity and making legacy choices while evolving architecture to avoid obsolescence. A strategy to encapsulate subsystems behind well-defined interfaces is currently being implemented to simplify maintainability.

Conclusion:

Maintaining a long-term Rails application like DNSimple is a substantial challenge, but it's also rewarding. Anthony Eden highlights the importance of learning from past design decisions, adapting to new technologies and frameworks, and fostering a collaborative environment as vital strategies for future success. The Rails community's ongoing evolution and support remains an invaluable asset for such journeys.

00:00:04.860 Hello, I'm Anthony Eden, the founder and CEO of DNSimple, a domain management automation platform for all your domain name and DNS needs. Today, I'd like to talk about what it's like running a Rails application for more than ten years.
00:00:10.860 First, I'm going to go through a brief history of DNSimple and the application itself. Next, I'll discuss the software design choices I made early on and how they have evolved over time as our team and the application have grown.
00:00:22.920 Next, I'm going to talk about the evolution of testing at DNSimple, followed by how we operate DNSimple. This has had a dramatic impact over time on both the design of the application and the system as a whole. Then, I'll cover a bit about dependency management – how we handled it early, how we approached it later, and how we manage it now.
00:00:39.300 I will also discuss some changes in our team over time and how these have impacted the design of the application and its ongoing operations. Finally, I’ll touch on the future and some of the challenges we still face today.
00:00:59.100 Let's start with a little bit of history about DNSimple. The DNSimple application is a Rails application that was first created on April 7, 2010, which marked the initial commit. In the beginning, there was only one developer working on the Rails application – that was me. The other person who helped create the business was focused entirely on the operations side, particularly on the DNS infrastructure.
00:01:22.560 From the initial commit to the launch, it took about three months to create a minimum viable product that included only DNS services. Throughout those first three months, I made several decisions based on the best practices of Rails at the time, and I am sure I made quite a few mistakes along the way – which is to be expected.
00:01:54.899 Let's take a look at some of those early software design choices and how they have changed over time. During those early days, I was following the Rails best practices that were widely accepted at the time.
00:02:12.780 The first example I'd like to discuss is the use of the 'thin controller, fat model' principle. While the thin controller concept is an excellent idea and still holds true today, the fat model approach, while manageable for smaller applications, becomes more difficult to maintain in larger applications.
00:02:48.620 Here's an example of the register method from one of the earliest versions of the domain model. This was used to register a domain name by taking the necessary extended attributes for the domain, creating an options hash, passing that to the purchase method in the upstream provider model, and then checking if the returned result included a valid order ID. If it did, it would set the registration status to 'registered,' save the local model, and return the result of that save; otherwise, it would return false and log the relevant errors.
00:03:38.760 Now, let's take a look at the controller corresponding to this action. If the register operation was successful, the application would deliver an email notifying the customer that the domain was registered. However, I wasn’t entirely adhering to the thin model concept, as complex business logic was already slipping into multiple places in the early versions of the DNSimple application.
00:04:41.460 Additionally, I was utilizing ActiveRecord callbacks fairly extensively. For instance, before creating a subscription in Chargebee, a callback would attempt to create it there and, if successful, set the Chargebee subscription ID; the opposite would occur for destruction. Meanwhile, the direct access to the ActiveRecord query interface was scattered throughout various portions of the code, which complicated understanding and maintaining query logic.
00:05:39.300 As a consequence, we faced several challenges with this approach. The use of God objects emerged, which made the models complex and large with many responsibilities. This led to difficulties in understanding and evolving the system due to the tightly-coupled methods and behaviors within these God objects.
00:06:47.460 In particular, the use of ActiveRecord callbacks further complicated our design. They often reached outside the model's intended area of responsibility, causing tight coupling with other system components, which resulted in a lack of clarity and ease of evolving the application.
00:07:53.640 To address some of these issues, we introduced the concept of commands. In traditional patterns, commands encapsulate an action and are designed to facilitate actions that can be undone and redone. Initially, I aimed to create commands in a way that would allow straightforward execution behaviors for various tasks, but over time, they became a means to aggregate business logic and provide integration across multiple components.
00:08:54.420 To illustrate, here is how the contact create command would be called today. The command execution takes a context as the first argument and accepts additional parameters afterward, which might hold information about the current account and user context. Instead of tightly coupling logic in the controller, the command encapsulates the operations, which enhances clarity and maintainability.
00:09:45.600 As we transitioned, we saw significant changes. For example, the command instance method now handles parameters more cleanly. The command implementation creates the contact directly, and upon success, sends notifications without cluttering the controller logic.
00:10:55.200 Another important addition to our architecture was the concept of finders. This is loosely based on the repository pattern, allowing us to have a single place to look up collections or instances of specific models. This approach ensures consistent methods for retrieving contacts across the system.
00:11:36.780 The evolution of finders is evident. Initially, we only pulled from users, but as we structured our application around accounts, we refined our approach, ensuring all entities were properly linked to accounts rather than users. This encapsulation facilitates easier access and ordering of data in a more streamlined way.
00:12:44.400 Now, let’s shift gears and examine the evolution of testing at DNSimple. In the earliest versions, we leaned heavily on Cucumber and RSpec for testing purposes, primarily focusing on acceptance tests rather than unit tests.
00:13:36.379 Here's an example of an early acceptance test demonstrating that a user can activate their account. It includes a description of the desired feature and outlines the scenarios to activate the account through the user interface.
00:14:04.920 However, as our system grew, we determined that maintaining Cucumber tests became too time-consuming due to frequent interfaces and application changes. So, we transitioned to using RSpec for both unit and integration tests.
00:15:06.240 Running the entire test suite can be slow, but it allows us to execute focused unit tests, even though there's room for improvement. This challenge is common for applications that continue to expand without a dedicated focus on unit tests.
00:16:14.280 Let’s discuss the operational evolution of DNSimple. In our early days, we utilized both PostgreSQL and MySQL. While PostgreSQL served as our primary data store for various models, MySQL was used specifically for storing our DNS server zones.
00:17:32.580 As we scaled, we transitioned to an Anycast system, distributing our authoritative DNS servers across multiple global locations and shifting from virtual private servers to bare-metal servers. This transition significantly redefined how we handle data distribution and storage.
00:18:14.280 The evolution of our background job processing saw us moving from Resque to Sidekiq and eventually to Sidekiq Enterprise. This required us to change our application to abstract direct dependencies on Resque, paving the way for a more flexible architecture.
00:19:59.820 In terms of our authoritative DNS, we adapted from a simpler operational model dependent on a few VPS instances, moving toward more robust implementations that enhance durability and performance.
00:20:54.900 Next, let's talk about dependency management. Any long-standing application with external dependencies must approach dependency management thoughtfully, and initially, we relied heavily on an ad hoc method.
00:21:44.100 Updating dependencies was a manual process without any structured methodology, leading to challenges when transitioning between different Ruby and Rails versions throughout our journey.
00:22:45.600 As we matured as a team, we began adopting a manual yet well-defined policy surrounding dependency updates, eventually transitioning into an automated approach with Dependabot to handle updates seamlessly.
00:23:55.560 Moving on to team changes, the impact of new hires on the application's evolution has been significant. The initial version was the sole effort of one developer, yet bringing in new talent sparked fresh ideas and suggestions for improvement.
00:24:56.760 As the team expands and contracts, we face the challenges of ensuring cohesion and consistency across the code. Each developer brings unique perspectives that contribute to evolving our application standards.
00:25:57.060 With varying levels of expertise among team members, we constantly provide opportunities for information sharing and knowledge transfer through pairing or even mob programming, enhancing our collaborative efforts.
00:26:56.700 While strong opinions and coding styles can spark diversity in thought, they can also create friction within the team. Navigating these complexities is important as we aim to maintain harmony within the growing codebase.
00:27:42.360 An additional challenge we continually address is consistency, particularly in documentation. Our early codebase lacked clear documentation, but over the years, we have increased our efforts to establish better documentation practices.
00:28:34.500 This has included developing a team Wiki for maintaining guides and ensuring every area of our application is well-structured and easily navigable. With over 27,000 commits and ongoing development, we strive to keep our system manageable.
00:29:38.520 As we look to the future, we continue to face challenges like managing legacy choices and increasing complexity. Each decision made today will impact the application's design and maintainability moving forward.
00:30:32.020 Managing complexity is a priority as we aim to break our systems into more discrete subsystems, which will remain cohesive within our Rails framework while fostering independent maintenance capabilities.
00:31:50.940 In closing, maintaining a long-term Rails application is a challenge, but it is also immensely rewarding. We benefit from the strong foundation of the Rails community and anticipate many more years of successful application development.
Edit
Explore all talks recorded at RailsConf 2021
+61