A Rails Performance Guidebook: from 0 to 1B requests/day

Summarized using AI

A Rails Performance Guidebook: from 0 to 1B requests/day

Cristian Planas • December 15, 2023 • Taipai, Taiwan • Talk

In this session presented at RubyConf Taiwan 2023, Cristian Planas discusses the importance of performance in application development, particularly for Ruby on Rails frameworks, illustrating that building features is not enough if those features are not performant. The presentation is structured around critical patterns, strategies, and real-life experiences at Zendesk—a company managing billions of requests daily.

Key points discussed include:
- The Challenge of Scalability: Cristian addresses the common concern regarding Ruby’s scalability, comparing it to Python and highlighting that many companies successfully use Rails for large-scale applications, like Shopify.
- Learning from Experiences: He shares his journey of dealing with scaling while leading the authentication team at Zendesk, reflecting on early misgivings about the capabilities of Rails and emphasizing that scaling challenges often appear later in application's development.
- Real Techniques for Performance Improvement:
- Error Budgets: Setting up error budgets is essential for identifying performance breakdowns.
- Database Indexing: The importance of efficient indexing in databases is underscored, suggesting that many performance bottlenecks stem from a lack of it.
- Data Sharding: Cristian recommends dividing data effectively to lower the load on any single database.
- Data Minimization: Fetching only necessary fields rather than using 'select all' commands to minimize the data size requested from databases.
- Archiving Strategies: Long-term storage solutions for less-accessed data can help maintain response times.
- API Endpoint Limits: Limiting API endpoints helps manage potential usage spikes that can lead to performance issues.
- Continuous Evaluation: Regular assessments of resource consumption can prevent unnecessary bloat in applications.
- Trade-offs in Performance Engineering: Emphasizing the need for informed trade-offs that accommodate business needs while balancing speed and complexity.

The session concludes by reinforcing the idea that a focus on developer happiness within the Rails community can nurture an environment where application performance can be optimized organically. Happy developers lead to better-performing systems.

A Rails Performance Guidebook: from 0 to 1B requests/day
Cristian Planas • December 15, 2023 • Taipai, Taiwan • Talk

#rubyconftw 2023

A Rails performance guidebook: from 0 to 1B requests/day

Building a feature is not good enough anymore: all your work won't be of use if it's not performant enough. So how to improve performance? After all, performance is not an easy discipline to master: all slow applications are slow in their own way, meaning that there is no silver bullet for these kinds of problems.

In this presentation, we will guide you across patterns, strategies, and little tricks to improve performance. We will do that by sharing real stories of our daily experience facing and solving real performance issues in an application that daily serves billions of requests per day.

RubyConf Taiwan 2023

00:00:28.960 Building a feature is not good enough anymore: all your work won't be of use if it's not performant enough. So how to improve performance? Performance is not an easy discipline to master; all slow applications are slow in their own way, meaning there is no silver bullet for these types of problems.
00:00:58.920 In this presentation, we will guide you across patterns, strategies, and little tricks to improve performance. We will do this by sharing real stories from our daily experiences facing and solving actual performance issues in an application that serves billions of requests per day.
00:02:27.879 Hello, Taipei! I'm really happy to be here. My name is Cristian. I'm from Barcelona, which is a nice city, and I currently work at Zendesk. Zendesk is a pretty big company, with around 1,000 engineers, and our architecture is still mainly based on Rails.
00:03:15.720 First of all, I wanted to explain why we are doing this presentation. As math says, performance skills also grow, but the question often arises: does Ruby scale?
00:03:50.640 In this session, some of you might have experience from computer science school. If you recall your first day, you probably remember learning about the various concepts that are crucial for software development. One reality is that scaling is an ongoing conversation and concern within our community.
00:05:10.560 Just a couple of moments ago, we were talking about how Ruby is often criticized for not scaling. However, if you check the benchmarks, Ruby’s performance is comparable to Python’s. Yes, we are a little bit slower, but it’s a similar situation. Interestingly, you don't hear the same concern being raised about Django.
00:06:23.199 The thing is that there are many companies using Rails at a large scale. In fact, it’s amusing to see many discussions on Twitter by people who have little idea about the subject, yet they frequently make claims.
00:07:20.479 For instance, one of my favorite tweets was someone claiming that Zendesk doesn't use Ruby at all. I've been working at Zendesk for 10 years, and I can guarantee you that Ruby is a significant part of our stack. So when someone claims that a specific company does not use Ruby, it is often misinformation.
00:07:41.800 Shopify, another major company, demonstrates the capabilities of Rails. Every Black Friday, they showcase incredible data on how Rails can scale efficiently. Last year, they served 76 million requests per minute. This emphasizes that Rails does indeed scale as needed; it’s a fact, not just a theory.
00:08:13.879 So the question remains: why do some people believe that Rails does not scale? One potential explanation requires a bit of time traveling. Let’s go back to the year 2010, when I was building my first application, which was a movie magazine.
00:09:24.120 I wanted to add authentication to my application. As many of you know, dealing with authentication can be complex, and I eventually transitioned into a tech lead role at Zendesk’s authentication team. However, during my early 20s, I had a lot to learn. I found a gem called Devise that made it easy to implement authentication in Rails.
00:09:55.519 The simplicity of Devise illustrated that even complex problems, like authentication, can be manageable for developers using Rails. However, what about scaling? When I was the Technical Leader at another company, we faced the same scaling challenges despite having just one engineer.
00:11:02.959 Through my experiences, I've found that scaling is an issue that emerges later in the development of an application. Initially, the focus is on building features. But as your application grows, you will need to address scaling, which can feel overwhelming.
00:11:22.079 While scaling can be challenging, it’s important to know that every application is unique, and common practices won’t work for every situation. It’s critical to understand that no silver bullets exist for performance problems.
00:12:40.079 I’d like to share techniques we've used to deal with performance issues at Zendesk, a customer support platform handling billions of tickets. As we scale, monitoring becomes crucial to understanding what's happening within the application.
00:13:31.519 I recommend setting up error budgets to analyze performance breakdowns and identify our acceptable response times. For instance, if an endpoint has a specific response time requirement, tracking this data is essential to maintain good service.
00:17:06.079 When discussing the aspects that are crucial for performance, I want to remind everyone about the importance of database indexing. Many performance issues arise simply from a lack of indexing, which can massively slow down processes.
00:17:36.600 Moreover, sharing in databases is something that should not be overlooked. By dividing data efficiently, we can improve performance greatly. Data sharding allows for focused access, reducing the load on any single database.
00:18:29.759 Another fundamental concept is to minimize the amount of data fetched by your application. It’s vital to select only the necessary fields rather than always using select all. Depending on your case, it may drastically reduce the amount of data transferred.
00:19:51.320 Also, pay attention to how information is stored long-term. Archiving older or less frequently accessed data into special storage solutions can help maintain a quick response time for live applications, keeping your queries efficient.
00:20:02.480 Now, regarding API design, pro-limiting your endpoints is essential. You need to anticipate potential spikes in usage since a lack of limits can result in severe performance issues down the line.
00:21:20.360 As you implement new features over time, be mindful of how they affect system performance. Regularly evaluate the resources that your application is consuming to ensure that it does not become bloated with unnecessary data.
00:22:15.400 The crux of performance engineering lies in making informed trade-offs and understanding the limits of what your system can handle efficiently. The best systems accommodate the specific needs of the business while maintaining a balance between speed and complexity.
00:23:00.240 In conclusion, let’s remember that a strong focus on optimizing for developer happiness within the Rails community fosters an environment where performance can grow organically. A happy developer is a productive one, leading to better-performing systems.
00:39:42.400 Thank you for your attention! Now we will have a Q&A session, so please feel free to ask any questions you might have.
Explore all talks recorded at RubyConf Taiwan 2023
+19