Talks
Keynote: Rails Doesn't Scale
Summarized using AI
Keynote: Rails Doesn't Scale
by Mark ImbriacoIn his keynote at RailsConf 2018, Mark Imbriaco explores the theme of scalability in technology, specifically addressing common misconceptions about Ruby on Rails and how it handles growth. He begins by introducing himself and sharing that the title "Rails Doesn't Scale" is intended as clickbait to stimulate discussion. Imbriaco distinguishes between performance and scalability, explaining that performance relates to the speed of individual requests, while scalability involves how well a system can handle increased load by adding more resources.
Key points discussed include:
- The Concept of Scalability: Imbriaco underscores that scalability often gets misinterpreted. Linear scalability suggests that by adding resources, system performance will double proportionately. He debunks the myth that Rails inherently lacks scalability, encouraging the audience to consider architectural choices over the framework itself.
- Socio-Technical Systems: He emphasizes the importance of the social aspect accompanying technical systems, suggesting that understanding both elements is crucial to effective scaling. This "socio-technical system" includes users, developers, and administrators, which he argues contribute more significantly to scaling than mere technical design.
- Personal Journey and Learning: Imbriaco shares anecdotes from his career, including his first encounter with programming languages like Perl and Ruby, and the pivotal moments that led him to Rails. His narrative includes worked experiences at companies like 37 Signals and Heroku, where he learned valuable lessons about building collaborative cultures.
- Community Sharing: One major takeaway is the idea that community is a critical resource that enhances scalability. When individuals share knowledge and experience, it produces a multiplicative effect on growth rather than a linear one. He encourages developers to contribute actively, as this can exponentially amplify the community’s capabilities.
- Lessons from Failure: He reflects on his entrepreneurship experience, stressing the need for careful management of workloads and responsibilities to prevent burnout, mentioning personal experiences that serve as cautionary tales.
In conclusion, Imbriaco argues that people are the driving force behind scalability, not just technology. The key takeaway is that engaging with the tech community and sharing knowledge amplifies the growth potential of systems, ultimately enabling a far greater impact on technology development and implementation than technology alone could achieve.
00:00:11.519
Hey everybody, I’m Mark Imbriaco. I hope you like my clickbait title. They asked me for a title for my keynote, and I tried to come up with the most obnoxious and annoying thing I could think of to say at RailsConf. It’s funny, but it also has a point.
00:00:24.080
Let me introduce myself. I have to say thank you to Pivotal first. I work at Pivotal, and I have this title: Field CTO of DevOps and Transformation. What that really means is that I talk to big companies and explain how the same things we were doing at 37signals, Basecamp, Heroku, and others can work for them.
00:00:38.160
I get this fancy title so they’ll actually listen to me. Just to manage expectations about this talk, it’s roughly half self-indulgent nostalgia for me and another half-ish of thought leadership. Maybe it’ll accidentally be insightful too; we’ll see. You can tell me afterwards if I was insightful at all.
00:01:06.320
So, let’s talk about scalability since my talk is theoretically about that. First, let’s define scalability. This term is often not well understood. The difference between performance and scalability is crucial. Performance is how fast you can go—can you meet a specific response time for a defined number of users? Scalability, on the other hand, means if I’m adding more users and have a higher throughput requirement, can I just add more resources, like another server or another Heroku dyno, to make it go faster?
00:01:20.640
This notion is interesting because it applies to many things. We could talk in detail about various models, like the Universal Scalability Law, which is fascinating, but we’re not going to delve into that right now. Instead, we’ll explore linear scalability. Linear scalability is what typically comes to mind when people say Rails doesn’t scale; they mean you can’t just add another server and expect to get double the throughput.
00:02:09.360
Plot twist: Rails actually scales! It turns out I wasn’t lying. But what we’ll explore together here is that linear scalability of technology is often quite boring. It’s an architectural concern. Anything can scale, and it’s really about how you design your system. Rails doesn’t preclude that at all; it never has. Even when Twitter claimed that Rails didn’t scale, they were mistaken.
00:02:47.120
I find linear scalability boring. What intrigues me more are the resources. If we acknowledge linear scalability where I’m adding servers, can we find ways to scale in a hyperlinear fashion? I believe the answer is yes, and that’s what we’ll discuss further. The reason I think linear scalability in technical systems is not particularly interesting is that scalability is a characteristic of the system as a whole. The systems we run comprise a technical system, like the Rails app we built, but that’s just part of the story.
00:03:29.760
In fact, the social system attached to the technical system is probably the least interesting part of the story. This includes your users, your system administrators, your developers—essentially all the social aspects surrounding and interfacing with your technical systems. When we discuss these components together, we refer to them as a socio-technical system, and this is what is most important to scale.
00:04:21.040
I personally believe that scaling on the social side is far more significant and interesting than on the technical side. So, story time! I’ll spend the rest of my time giving you a hopefully interesting historical presentation about my life, particularly my work life, and the insights I’ve gained along the way. I'll try to keep it remotely interesting.
00:04:52.080
I picked 1992 as a significant date to kick off this story because that's the year I graduated high school and began college. It’s interesting since it was when I first got involved in systems. My first job during college was in the fall of my freshman year in the CS department as a system administrator. So, I’ve been doing this for about 25 years—quite a bit longer than the 20 years David mentioned, and I’m underselling it. But let’s split the difference.
00:05:22.479
My story, like many historically significant stories, begins with a camel in 1992. In fact, I was quite passionate about camels. Perl was the first programming language I didn’t just learn but truly fell in love with. There were many things about Perl that I appreciated; a lot of the same features I admire in Ruby today were present in Perl. It gave me the power to be expressive and allowed me to work in a way that felt natural.
00:05:58.080
At that point in time, Perl was basically the natural language of the web. If you were doing anything online back in '92, congratulations! You were ahead of the curve because inline images didn’t exist until 1993. I looked it up earlier—I thought I was right: Mosaic 0.3 came out mid-1993, and that’s when we finally got inline images, which was groundbreaking.
00:06:30.880
I loved Perl partly because of the ethos surrounding it, which Larry Wall exemplified through the 'There's more than one way to do it' philosophy. This ethos fostered a sense of community. There was a Perl channel on IRC, which symbolized the online community back then. It was relatively toxic and certainly less friendly than communities today—but it was a community nonetheless.
00:07:05.040
Even though the Perl community was challenging, I found it exciting. In 1992 and 1993, working on computers made me feel like a magician. There was limited documentation; accessing books was difficult, and simply getting online was a significant achievement. I was a computer science major, bending the computer to my will, writing complex coding that only I could decipher—all while loving every second of it.
00:07:55.680
After dropping out of college in 1994, I founded a web company. Interestingly, during the early '90s, we spent more time explaining to people what the web was rather than making money. But I was deeply involved with mod_perl, which really represented my first significant foray in web development. I was very active in this community and even ran the mailing list for mod_perl for a long time. I attended the first few Perl conferences and spoke there, which was a wonderful experience and a great time in my life.
00:08:48.240
However, eventually, when you find yourself explaining the web too much without actually earning money, life catches up fast. I had to grow up and focus more on practical aspects to provide for a family. Leaving that startup led me to work for a bank, which I don't recommend due to how uninspiring it was, but I learned a lot there.
00:09:20.640
The key lesson from that experience was that you need to care about what you do. Just earning a paycheck isn’t enough. I didn’t last long, eventually moving on to another job at AOL for a couple of years, which sounds insane but was, in reality, quite thrilling. I was running one of the largest websites on the planet with an unlimited budget, and we had a 100 megabit connection to my desk. It was an unbelievable era in the late '90s.
00:10:00.960
In the midst of all this, I left AOL because my family mattered more to me than remaining there, which was something that Ernie had also mentioned earlier. AOL had a rigid culture demanding physical presence in the office, which was in Northern Virginia—a place I personally didn’t enjoy. Ultimately, I desired to be closer to family, and that decision led us to move back, embedding certain realities into my life.
00:10:55.680
Then life progressed even faster. Facebook had the mantra 'move fast and break things,' but for me, I had four kids and needed to ‘move fast’ with stable infrastructure. That led me to what I now refer to as my 'Java years.' Sure, they’re often laughed at, but I’m genuinely grateful for them as they provided stability during the Dot-Com crash when many were struggling.
00:11:56.720
During that time, I was developing software for estimating repairs on heavy trucks—far away from the glamorous side of coding, but I kept my head down and appreciated the opportunities I had. It allowed me to work remotely, which was beneficial. However, I grew to dislike coding, an unfortunate realization at that point in my life.
00:12:31.840
To reignite my passion, I decided to attend a conference called No Fluff Just Stuff, which was somewhat elitist but intriguing. Their pitch was to focus on things that would empower attendees to improve as developers. Then one day, I attended a talk by Dave Thomas. He was well-known in the Java circles and started discussing Ruby. I remember sitting in the front row, utterly mesmerized when he started demonstrating, realizing that Ruby gave me the same feelings I got when I first discovered Perl.
00:13:50.480
I was seeing Perl through the lens of Ruby, feeling a spark of inspiration. I realized I could do things with Ruby that I loved about Perl, but with structured programming and objects, which was valuable to me at that stage of my career. I had witnessed numerous discussions around Rails since it was going through some intense attention back in 2005. I went back to my hotel room that night and built my first Ruby on Rails app.
00:14:46.560
I was not exaggerating when I say that when I first saw Ruby, it felt like home. In fact, after seeing Dave's talk, I returned home and convinced my boss to allow me to start a new project using Rails, despite only knowing it for two weeks. It was a crazy success. In just about a month, I persuaded them to let me rebuild an existing project into Rails, which took about four years of work, but we accomplished it in just four months.
00:15:29.520
That company was called Decisive, and I want to specifically acknowledge them for the faith that the CTO, Satish Joshi, placed in me. It truly changed my life, leading me to work at 37signals, which set the stage for my ongoing career. They are still using Rails today, and practically the day I gave that talk, someone from our community was hired as their VP of Engineering.
00:16:23.200
As we finished deploying this new version of the app, things escalated quickly. I tend to be pretty excitable, and anyone who knows me understands that I can be very long-winded. 37signals was looking to hire, and I read this job posting—it felt personally tailored for me! I was vacationing with my wife when I came across it, and I felt an irresistible pull to apply. I wasn’t even looking for a job; I was genuinely satisfied for the first time in years with my work in Rails.
00:17:21.840
However, upon seeing that posting, I realized I had to go for it. I quickly emailed David, explaining why he should hire me. I didn’t have a traditional resume, but I did send him 1300 words detailing my motivations, despite knowing I wasn’t actively on the job market. He must have been bewildered by that length, and I cringed upon rereading my email.
00:18:00.080
Still, let’s look at the timeline because it’s quite interesting. I sent that email on August 17th, and by September, a month later, I was officially part of 37signals. This moment changed my life—and I don’t say that lightly. Almost everything thereafter in my career, I owe to my time there, and to David, Jason, and our entire team, not just for the technical prowess but for the community we built.
00:18:52.960
Ruby on Rails was a huge aspect, but it was the ethos of the community and the kindness amongst those in the Rails community that really resonated. David's presentation this morning truly touched on these values—broadening the base, inviting more people in and making them feel empowered to create amazing things. Those lessons learned from 37signals still linger with me, even though we were often uncertain about what we were doing back in 2006.
00:19:38.720
We had a strong initial framework where you could create a blog in 15 minutes and potentially get it functioning on a hosting provider within days. However, we still had a significant learning curve ahead. I identify as a mediocre developer who's good at problem-solving despite the challenges we faced early on. We were essentially putting together a new technology with a minimal playbook.
00:20:27.760
Ezra, one of the founders of Engine Yard, was instrumental during this time as we were sharing and disseminating knowledge about deploying Rails apps. As he and I were synonymous with the discussion around Rails and deployment procedures, we educated our peers and suppliers about simplifying the process of running Rails apps. However, we needed the community to figure this out together; we needed to compress this knowledge and make it easily accessible.
00:21:28.520
Of course, during this journey, we often faced skepticism—why did Twitter fail to scale? Why was Rails too difficult to deploy? These challenges highlighted always targeting issues and failing to recognize that we were offering open, candid advice, expecting that adults understood the nuanced nature of our technology and its challenges. Yet, that conversation wasn’t isolated to just the Ruby community—there was a culture of niceness surrounding it. The lessons from our community on being gracious were paramount.
00:22:27.440
The simple ethos was about building community. Thus, when we engage and share resources in our community, it bears the fruits of multiplicative impact rather than the linear 1x lift. Contributing to your community isn’t just an isolated act; it has ramifications that lead to growth on a larger scale. Everyone here, through your contributions, fosters that hyperlinear growth we’re after.
00:23:43.440
My four years at 37signals were remarkable, but eventually, I decided it was time to move on. I discovered that my passion lay in enabling people—especially developers. I had been perpetuating the notion that 'software is eating the world' for a while, emphasizing the power of technology to effect change. This notion still resonates today; enabling others through software is where my heart lies.
00:24:33.920
In 2010, I joined Heroku. It hit me deeply as it elegantly addressed numerous issues with deploying Rails applications. After some time, I transitioned to Living Social at Chad Fowler's urging due to the community surrounding it. Eventually, I found myself at GitHub, where the lessons of enablement were echoed again, just as they were at Digital Ocean. The key takeaway: the principles we stood for back in 2006 are still relevant and have evolved in today’s context.
00:25:45.600
When I left DigitalOcean, I agitated on founding my own startup, operable, for many of the same reasons. The spur came when a friend, James Gulick, who had been a significant actor in our community, unexpectedly passed away in an accident. The fragility of life struck me hard, prompting me to evaluate how I wanted to impact the world—relaying the message that one must truly care about their work.
00:26:29.440
When I finally left my job, I ventured wholeheartedly into this startup, raising money quickly, and building something new. However, we recently closed the doors—what many would call a failure. Still, I find great value in that experience. Failure can teach you deeply, and in my case, I learned especially about the importance of foundational lessons that had previously been instilled in me.
00:27:32.720
These lessons ranged from combating hero culture to building minimum viable products and cultivating feedback loops. It’s a realization that those who've been taken through the school of hard knocks don’t always internalize these lessons right away. Our past experiences underscore the importance of sharing knowledge, just as teaching others both aids them and reinforces your understanding.
00:29:00.320
When you enlighten someone, it unlocks profound effects. You don’t just teach; you sow the seeds for their own learning journey. Hyperlinear scaling comes from this communal growth and resource sharing. Everyone present in this room, every single person is essential for scaling our socio-technical structures—whether that be technology, community resilience, or the development of relationships, you contribute to shaping positive change.
00:30:42.960
In the end, let’s underscore that it’s not Rails, not any specific technology that primarily drives scalability—it’s people. It’s the collective effort of this community that makes technology scalable. People are the core element behind successful systems. Thank you!
Explore all talks recorded at RailsConf 2018
+98