How to Scale a Ruby Webservice

Jeremie Castagna

Jeremie Castagna

1 talk

How to Scale a Ruby Webservice

by Jeremie Castagna

Summary of 'How to Scale a Ruby Webservice'\nIn this session, Jeremie Castagna from AT&T Interactive discusses his experiences with scaling Ruby web services, especially focusing on the common misconceptions regarding Ruby's performance and scalability. The main goal is to demonstrate that with appropriate strategies, Ruby applications can efficiently handle a high volume of requests without the need to switch to more traditionally regarded faster languages like Java.\n\n#### Key Points Discussed:\n- Background and Context:

Jeremie starts by sharing his role at AT&T and how their services team initially built applications using Ruby on Rails. \n - He highlights Ruby's long-standing reputation for being less scalable compared to alternatives. \n\n- Ruby Performance Myth:
The speaker emphasizes that the programming language itself (like Ruby) is often not the bottleneck in web applications.
With proper architecture, a Ruby service can effectively work as a gateway to backend services. \n\n- Memory Consumption:
Acknowledges the significant memory usage of Rails applications, often constraining scalability on servers with limited RAM. \n - He shares insights on how unnecessary features can bloat applications, leading to inefficient memory use. \n\n- Strategies for Optimization:
Utilization of profiling tools to find and remove unused code, thereby optimizing memory and IO performance.
Recommendations to limit gem usage and potentially shift to lighter frameworks like Sinatra for service-oriented architectures. \n\n- Concurrency in Ruby:
Discusses the limitations of the Global Interpreter Lock (GIL) in Ruby that hinder true concurrency but explains how efficient IO operations can still facilitate multi-threading capabilities. \n\n- Operational Best Practices:
Highlights the importance of logging, monitoring, and implementing connection pooling and timeout limits to prevent resource bottlenecks.\n\n- Conclusion:
The session wraps up by reinforcing that by adopting memory optimization techniques, streamlining application code, and effectively managing threads, developers can significantly scale Ruby web applications and improve user experience.

00:00:04.930 Hello everyone, my name is Jeremy, and I've been with AT&T Interactive for four years now.

00:00:06.990 I started on their website team when it was a very small group, but it grew quickly. Now, I'm on the application services team. Our primary responsibility is to rebuild service applications for the website and mobile consumers. Essentially, anyone who wants access to Yellow Pages data must go through us.

00:00:29.939 When our services team first started, which was around four years ago, Ruby on Rails was considered suitable for everything. Naturally, the service was built using Rails, and it worked just fine until we began receiving a high volume of requests.

00:00:46.649 I wanted to share my experiences about working on a web service in Ruby and how we could scale it without rewriting everything in another language. The goal is to handle fewer hundred thousand requests to millions of requests.

00:01:02.550 Currently, our application handles upwards of two hundred thousand requests daily. One important point I want to emphasize is that the programming language used for your web service, in our case Ruby, is probably not the core issue. We prefer Ruby because of our familiarity, and if you have done the architecture correctly, your web service is essentially just a gateway to other backend services.

00:01:35.340 You shouldn’t need to spend too much time in the service you write; it should ideally pass through requests efficiently. I personally find Ruby to be an excellent language. It’s easy to use and has a gentle learning curve, making it easy for new developers to catch up quickly. Ruby is one of the faster interpreted languages available.

00:01:54.430 Just because the language isn’t the bottleneck doesn’t mean the way you use it isn't critical. So, what does a typical Ruby web app look like? Most Rails applications can appear quite similar on the surface—nice interfaces, but there’s often a lot going on underneath. For instance, we reached a point where just initializing a Rails app consumed about 350 MB of memory. On a server with even eight gigabytes of RAM, that limits the number of instances you can run. Many Rails applications are single-threaded and can handle only a limited number of requests simultaneously, leading to bottlenecks.

00:02:42.430 Furthermore, although HTTP is a powerful tool, it’s often underutilized. When communicating with other services, you don't want to halt your application's processes while waiting for responses. Here's the scenario: your application receives a request, and you have to fetch resources from the backend. This can lead to delays as your app waits for responses while other requests pile up.

00:03:04.300 Typically, visualizing server usage reveals a bottleneck where your application consumes a large amount of memory without effectively utilizing CPU resources. The common practice in Rails for scaling is to deploy multiple instances on the server, which can quickly become expensive, especially under high loads.

00:03:30.949 To address this, we need to reduce the RAM footprint of our application so we can handle more requests while keeping Ruby busy with meaningful work. This involves optimizing memory use and improving IO performance.

00:04:02.680 One approach is to streamline your application code and remove unnecessary features. How many of you have worked on a project for more than a year? As the project evolves, legacy features may linger without being evaluated for their current utility, leading to unnecessary bloat.

00:04:51.970 To help with this, we've implemented profiling tools to check which aspects of our application are actually being utilized. Tools like Rubygems help us identify unused classes and methods, allowing us to streamline our code further. Additionally, we found that Ruby classes consume significant memory. For example, creating numerous short-lived classes can lead to inflated memory usage that isn’t reclaimed effectively.

00:05:30.600 To manage memory more efficiently, we should avoid singletons and monkey patches, particularly if they involve dynamically creating classes during runtime. This can lead to rapid memory consumption. We can also benefit from reusing regex patterns by assigning them to constants rather than creating new instances repeatedly.

00:06:03.600 Another best practice is to limit the gems used in your application. While Rails is fantastic for HTML and form building, it may not be the best choice for service-oriented architectures that primarily deal with JSON or XML. Instead, focus on lightweight gems specifically for JSON parsing to minimize memory consumption.

00:06:59.020 Moreover, consider utilizing alternatives to Rails for your web services, such as Sinatra or other frameworks that allow you to efficiently manage requests with lower overhead. For instance, by moving away from Active Record for certain parts of your application, we adopted the Sequel gem, which provides database interaction with reduced overhead.

00:08:02.480 While still allowing us to take advantage of the helpful features from Active Support, we’re able to minimize unnecessary dependencies and optimize our application further. By employing these strategies, we have observed significant improvements, including reducing our request processing time and increasing request per minute (RPM) capabilities.

00:09:10.460 When implementing multi-threaded applications in Ruby, it's important to note that Ruby does not offer true concurrency due to the global interpreter lock. However, you can achieve concurrency by utilizing IO operations effectively, as Ruby allows threads to be released while waiting for IO requests.

00:10:08.890 Particularly starting with Ruby 2.0, the IO performance improvements have enabled our server to handle more connections efficiently by spinning up multiple threads per process. Each thread can manage its own IO operations simultaneously, significantly improving request handling capacity.

00:10:57.920 Utilizing a well-structured thread pool management system is crucial, as it reduces thread creation overhead while maintaining responsive application performance. By carefully managing threads this way, we can scale our applications efficiently and effectively.

00:12:45.270 We also have to ensure that our application does not hang indefinitely while waiting for responses from backend services. It’s essential to establish limits for your timeouts and to implement connection retry logic without allowing one slow service to degrade the performance of the entire application.

00:14:55.000 When connecting to external services and databases, utilize connection pooling to limit the number of open connections, preventing overload of your backend systems. Additionally, avoid holding on to slow connections and instead consider terminating them if necessary to keep other requests active.

00:18:40.650 In terms of operational practices, maintaining proper logging and monitoring of your services is crucial to diagnose potential issues before they escalate. By being proactive about resource management and connection handling, we can create scalable and maintainable Ruby web services.

00:20:14.365 To summarize, we’ve discussed how to scale Ruby web applications through reduced memory usage, optimized IO operations, and using thread pooling effectively. By leveraging efficient practices and tools to streamline processes, we have successfully increased our application's request handling capabilities.

00:22:06.920 Our goal has been to reach a state where user experience is unhindered by the application’s performance. The strategies discussed in this talk can help any Ruby developer enhance their web services and scale them to new heights.

00:23:48.300 I appreciate your attention and hope that these insights will be useful as you continue to develop web services in Ruby. If you have any questions, please feel free to ask them now.

LA RubyConf 2012