So Long, Hoboken: Migrating a Huge Production Codebase from Sinatra to Rails

by Eric Weinstein

The video titled "So Long, Hoboken: Migrating a Huge Production Codebase from Sinatra to Rails," presented by Eric Weinstein at RailsConf 2015, focuses on the complexities and strategies involved in migrating from the Sinatra framework to Ruby on Rails. The speaker underscores that software is inherently dynamic and migration efforts are essential yet challenging.

Key Points:

- Migration Concerns:

- Migrations should not be seen as a silver bullet for all problems; careful planning and execution are crucial.

- The best-case scenario might often result in no noticeable changes for users, making it a tough sell to stakeholders.

- Reasons for Migration:

- Address code complexity, tooling issues, security, and maintenance challenges.

- Improve application performance as Sinatra applications can suffer from overloads and bugs.

- Insights from a larger community of Rails developers can aid in overcoming hurdles that arise during development.

- Hiring and inter-team training is simpler with a more standardized Rails framework compared to Sinatra.

- Architectural Considerations:

- The migration included adopting a service-oriented architecture utilizing Backbone and React, maintaining Java for backend services.

- Key focus was placed on fault tolerance and isolation in the application’s architecture.

- Migration Methodologies:

- Melting Approach: Mount Sinatra apps in Rails, enabling gradual integration but causing performance overhead.

- Strangle Pattern: Incremental migration allows for phased transition while developing new applications.

- Testing and Automation:

- Importance of thorough testing with tools like RSpec and CI integrations to ensure a seamless migration.

- Automate the development process to maintain integrity during deployment.

- Implementation Steps:

- Start by converting service classes into Rails models and mapping routes carefully.

- Sequential changes in views and phased removal of deprecated components were also crucial steps.

- Concluding Insights:

- Successful migrations demand ongoing communication and adjustment within teams, reflecting Conway's Law.

- Consider the user experience throughout the migration process to ensure growth aligns with user needs.

In summary, Eric Weinstein emphasizes that while resetting the underlying architecture can alleviate poor coding practices, the migration itself requires a deep understanding of both technical and human factors for effective execution.

00:00:11.639 Let me start with my obligatory social media contact info. You can find me on Twitter. I know it's hard to read on the slide.

00:00:22.800 And as you can see on the slide, the gold medallion signifies that I am a king-certified Ren Faire participant. I attended the Tuxedo Park Renaissance Fair last summer.

00:00:31.599 So, I started working with Rails about three years ago, but I had to take some detours through Sinatra. Now I'm coming back to Rails, and it has been an interesting journey that I believe relates to migration.

00:00:41.920 If you enjoy reading, I wrote a book late last year titled 'Preview Wizardry'. The great folks at Nostalgia Chat have set up a promo for it.

00:00:50.320 If you're interested, you can visit the Nostalgia site and use the RailsConf promo code for a 40% discount.

00:00:54.879 If you are familiar with the book, please come find me after the show. If you don't like it, please direct all complaints to my fellowship.

00:01:50.320 Now, onto the talk. I must share an obligatory caveat: There are no silver linings. A migration is not a cure or a do-over button.

00:01:57.040 You should always be skeptical of anything that promises to fix all your problems, whether it’s a new language, technology, framework, or method.

00:02:07.280 Migrations and rewrites can be really fun, but they're also really hard. They should never be undertaken lightly.

00:02:12.800 It’s very easy to put yourself in a corner or start rewriting without a clear direction. Therefore, planning and thoughtful execution are essential in the migration process.

00:02:30.800 Now, you might be wondering why we would want to undertake something so dangerous. When talking to business stakeholders, I often find myself explaining that if I do everything perfectly, the best outcome may be that nothing changes.

00:02:42.160 This is a hard sell; it’s tough to say that the best thing that could happen is that nothing changes from the user's perspective.

00:02:54.959 However, there are a number of code-related reasons to consider migrating: complexity reduction, improving tooling, enhancing security, and maintenance, all of which we often overlook while coding.

00:03:17.519 Another reason is application performance. We often hear criticisms about Ruby on Rails applications regarding their performance. It's worth reconsidering how we structure our Rails applications compared to our Sinatra applications.

00:03:36.879 In our case, we experienced a lot of overloading and reloading, so moving to a new codebase with Rails could improve that.

00:03:50.959 The multitude of bugs, features, and issues within our previous Sinatra application was significant. The knowledge base around Rails, including insights from DHH (David Heinemeier Hansson), is much more extensive and understood compared to Sinatra.

00:04:14.560 Another reason to migrate is community development. We encountered difficulties working with open source projects using Sinatra because it has fewer conventions than Rails, which hampers tooling development.

00:04:31.840 In addition, since Sinatra isn't as widely used or maintained, it became challenging to find necessary resources. Development support, literature, and client education are much more robust when working with Rails.

00:04:50.560 There’s a plethora of Rails tutorials and resources, providing a more informed decision-making process when developing applications.

00:05:05.520 Additionally, hiring Ruby developers who are familiar with Rails is easier than finding bespoke Sinatra developers. A Rails application tends to have less variability, making it easier for newcomers to get acclimated.

00:05:18.240 Ultimately, every tech problem is often fundamentally a people problem. We realized that moving to Rails would help address many of these people-related issues.

00:05:31.360 Now, let’s discuss the architecture we adopted at RTR. It utilizes a service-oriented architecture with Backbone and React.

00:05:40.240 Currently, we’re working on incorporating React components into production. One major part of our transition is migrating from Sinatra to Rails.

00:05:56.800 Our services are built with Java, making them delightful to write applications with.

00:06:04.800 Although it’s surprising to hear someone describe Java and delightful in the same sentence, I stand by it!

00:06:14.960 In terms of data storage, we’re using MySQL and MongoDB, each offering unique advantages.

00:06:27.520 Our Rails applications weren't just one singular product but a suite of applications, making significant architectural decisions necessary for scalability.

00:06:43.840 For instance, we needed to focus on fault tolerance and isolation in our application design.

00:06:56.720 As we began examining our architecture, we discovered that our segregation of components wasn't functioning as intended.

00:07:10.760 A key challenge was that many front-end components were operating off the same backend services, leading to critical failures whenever one service became unavailable.

00:07:25.760 We realized that we hadn't achieved the desired level of fault isolation with our current architecture, prompting the decision to migrate everything to Rails.

00:07:36.160 Now, if you're considering migrating, there are a couple of different methodologies worth discussing.

00:07:48.000 The first is a 'melting' approach, where you mount all of your Sinatra applications inside Rails as separate units. This allows you to maintain legacy behavior while transitioning.

00:08:02.080 This method has the advantage of allowing you to keep your existing code separate while integrating Rails into the application.

00:08:19.440 However, with this method, you face performance issues as both Rails and Sinatra applications will run concurrently.

00:08:34.239 Each of the Sinatra applications will need to load repeatedly, causing unnecessary overhead.

00:08:48.480 The second approach is adopting a strangle pattern, which allows for incremental migration.

00:09:02.560 In this method, you create new Sinatra applications to handle aspects of the existing system as you carve out components for the new system.

00:09:18.720 It's a way to gradually transition pieces of your legacy application to a new architecture steadily.

00:09:31.360 This strangle pattern also gradually allows the team to adapt to new structures and components.

00:09:44.720 While it's significant progress to adopt this new methodology, you have to be cautious about not stopping too soon and leaving parts unfinished.

00:09:57.440 You should be vigilant, as it’s easy to feel like you’ve completed the majority of the work but may also have more left to tackle.

00:10:10.480 The third step is to focus on writing thorough tests. Exemplary test coverage is paramount for a successful migration; without it, you’re essentially flying blind.

00:10:21.920 At RTR, we utilize various tools for testing, such as RSpec for integration tests and Continuous Integration with Jenkins.

00:10:38.400 Throughout our migration process, our focus was on enhancing our test coverage to guarantee that components worked seamlessly together.

00:10:53.440 Our GPA on Code Climate was a mere 0.67 when I joined, but it improved to 2.1 by the time I left RTR.

00:11:07.680 Another critical step is applying significant automation throughout the development process.

00:11:22.520 When pulling requests are made to GitHub, Jenkins initiates tests and runs them before merging changes.

00:11:35.920 This process ensures all updates are validated before they affect production, which is essential for maintaining a steady deployment cycle.

00:11:51.280 Moving onto the models, we began our migration by converting our service classes into Rails models.

00:12:07.680 The goal is to streamline how services interact with Rails, making it easier to manage application state.

00:12:22.080 As the process continues, we aim to implement Rails gems that offer incremental improvements.

00:12:35.760 Next, we focused on incorporating Rails controllers to map existing routes from our Sinatra application.

00:12:48.000 This transition means we needed to create a detailed mapping of our current routes to the Rails structure.

00:13:01.440 As we established routes, we documented everything to ensure clarity in how requests would be handled in the new architecture.

00:13:16.560 For views, we needed to rearrange files to suit Rails’ expectations and to account for how Rails processes templates.

00:13:29.680 This phase was deceptively tricky, as it created opportunities for typos or mishandlings that could lead to failures deep in the application.

00:13:45.680 As such, we ensured thorough testing around these areas, adding checks to our integration testing whenever we identified issues.

00:14:01.440 Finally, we phased out deprecated components that did not align with Rails standards to streamline our updates.

00:14:12.080 Issues like duplication and security must be addressed during this process, ensuring the application's integrity while optimizing performance.

00:14:25.520 I can’t stress that this is an ongoing process, and it's essential to continually refine your strategies as you transition.

00:14:38.560 What we learned through this migration is the importance of clear communication during the software development process.

00:14:53.760 There's a truth to Conway's Law: the architecture of your application mirrors the communication structures within your organization.

00:15:08.440 As you work on your migration, think critically about how teams interact and how that will influence the new architecture.

00:15:24.680 There will always be more work than expected in the later stages of a migration, so prepare for ongoing refinements.

00:15:39.440 As you wrap up, remember that tech migrations are often solutions to poor architecture, and it necessitates understanding the underlying people problems too.

00:15:54.720 When thinking about migration, always consider keeping the user in mind and aiming to grow and adapt the application to meet their needs.

00:16:09.680 And that's all I have for you today. Thank you for your time!