00:00:16.080
Hello, everyone! Thank you all for being here. It's truly great to be a part of this conference, one of the best I've had the fortune to attend in my career. The experience has been fantastic so far, and we're almost through the agenda.
00:00:30.640
My talk today will be about our journey over the past 11 to 12 years with Ruby on Rails and our experience working with a monolith. I'll share why we initially built microservices, why we later chose to consolidate them back into a monolith, the success of this move, and the lessons we learned along the way, including the role Ruby on Rails played in this process.
00:00:39.280
It's a relevant topic right now, especially given the recent discussions on Twitter. I noticed DHH talking about monoliths versus microservices, and while that discourse may shift towards test-driven development soon, it feels topical, so I'm excited to share our experience.
00:01:03.559
Like many businesses, ours started with a useful service that led to growth. As we scaled, we faced challenges such as maintaining product-market fit, adding quick workarounds for major customers, keeping tests running, and ensuring deployments didn't crash the system. All of these issues contributed to a decline in productivity, leaving leaders reminiscing about the early days when their focus was solely on building products for customers.
00:01:42.799
As frustrations grew, discussions among engineers began to revolve around inconsistent coding standards. We faced outages and found security upgrades increasingly difficult. Various solutions were proposed, one of which was the shift towards microservices, which seemed like a popular trend following many conference talks. This concept of breaking up a monolith soon turned into serious discussions in informal settings and gradually became part of our roadmaps, championed by ambitious engineers eager to contribute to the company's future.
00:02:05.119
I work for Intercom, a B2B SaaS company that serves as a customer service platform, sending approximately 600 million messages monthly to over 800 million active users. We launched in 2011 amidst a wave of businesses capitalizing on emerging technologies like cloud computing, mobile, and of course, Ruby on Rails. This talk will cover our experiences with architecture across different phases that reflect our growth journey.
00:02:39.000
Our architecture has evolved from a single, small GitHub repository running on Heroku—a straightforward platform for shipping code and delivering services to customers—to a more complex system. As we began to outgrow Heroku's capabilities, we migrated to AWS, incorporating MySQL databases and custom deployment tools, which allowed us greater control and scalability. This period marked the pivotal moment when Intercom hit product-market fit.
00:03:28.080
The second phase of our journey began when I joined Intercom. We faced the dual challenges of scaling our team and infrastructure while simultaneously managing high ambitions for the company's growth. Our original team that had built the Rails application was great, yet we aimed to build even more robust products with a larger team. Conflict arose between maintaining our legacy systems while also innovating at scale.
00:04:54.600
Our engineers began experimenting with new patterns and technologies, but we faced major hurdles as our Rails app struggled with scalability issues. Challenges included MySQL database limitations, continuous integration nightmares, and a desire to keep our shipping cycle tight, which we believed was crucial for delivering value to customers.
00:06:02.720
In the face of increasing complexity, we sought to standardize our architecture, which began to expand beyond the confines of our monolithic Rails app.
00:06:16.600
Simultaneously, we built a Java application to handle our websocket traffic due to its high volume and complex load. Our Rails monolith communicated with this application via HTTP, and we adopted various AWS services that helped distribute workload efficiently. During this period, we also explored new options, including serverless architectures, though our experiences were mixed due to developer experience and maintainability issues.
00:07:43.000
As our architecture expanded, we continued to decouple various elements. We extracted the billing aspect into a separate Rails app to improve agility and minimize risk. This allowed the team to operate independently without affecting our core application's performance. Around this time, we began building additional services in standalone applications for new features that had significant overlap with existing ones.
00:09:13.000
The decision to build standalone apps continued, but as we evolved, we realized there could be an opportunity to bring certain services back into the monolith. This realization led us to a new appreciation for what we were achieving inside our monolith, heralding a shift in our thinking as we began the process of consolidating rather than fragmenting our architecture.
00:10:50.000
The third phase began as we started rewriting parts of our system, such as the user and company data interfaces, transitioning from MongoDB to DynamoDB for improved scalability. Through these revisions, we discovered the value in fine-tuning our monolithic architecture, consolidating various applications and their respective functionalities—ultimately driving a decrease in the complexity we had previously created.
00:12:00.320
Moving forward, we invested more in monitoring and observability, allowing our teams to focus on delivering quality features for our customers rather than maintaining numerous separate services. This investment has contributed positively to our operational efficiency.
00:12:30.000
In conclusion, while we initially sought to fragment our services for various reasons, we ultimately learned that there are substantial advantages to a well-structured monolith when it comes to scaling and operational efficiency. Intercom continues to grow, and as we evolve, we'll be focusing on ongoing improvements to ensure our architecture supports our objectives effectively.
00:14:00.000
Thank you all for listening. Feel free to reach out to me on Twitter @BrianScanlan or find me online. Your attention is greatly appreciated.
