00:00:04.930
Hello everyone, my name is Jeremy, and I've been with AT&T Interactive for four years now.
00:00:06.990
I started on their website team when it was a very small group, but it grew quickly. Now, I'm on the application services team. Our primary responsibility is to rebuild service applications for the website and mobile consumers. Essentially, anyone who wants access to Yellow Pages data must go through us.
00:00:29.939
When our services team first started, which was around four years ago, Ruby on Rails was considered suitable for everything. Naturally, the service was built using Rails, and it worked just fine until we began receiving a high volume of requests.
00:00:46.649
I wanted to share my experiences about working on a web service in Ruby and how we could scale it without rewriting everything in another language. The goal is to handle fewer hundred thousand requests to millions of requests.
00:01:02.550
Currently, our application handles upwards of two hundred thousand requests daily. One important point I want to emphasize is that the programming language used for your web service, in our case Ruby, is probably not the core issue. We prefer Ruby because of our familiarity, and if you have done the architecture correctly, your web service is essentially just a gateway to other backend services.
00:01:35.340
You shouldn’t need to spend too much time in the service you write; it should ideally pass through requests efficiently. I personally find Ruby to be an excellent language. It’s easy to use and has a gentle learning curve, making it easy for new developers to catch up quickly. Ruby is one of the faster interpreted languages available.
00:01:54.430
Just because the language isn’t the bottleneck doesn’t mean the way you use it isn't critical. So, what does a typical Ruby web app look like? Most Rails applications can appear quite similar on the surface—nice interfaces, but there’s often a lot going on underneath. For instance, we reached a point where just initializing a Rails app consumed about 350 MB of memory. On a server with even eight gigabytes of RAM, that limits the number of instances you can run. Many Rails applications are single-threaded and can handle only a limited number of requests simultaneously, leading to bottlenecks.
00:02:42.430
Furthermore, although HTTP is a powerful tool, it’s often underutilized. When communicating with other services, you don't want to halt your application's processes while waiting for responses. Here's the scenario: your application receives a request, and you have to fetch resources from the backend. This can lead to delays as your app waits for responses while other requests pile up.
00:03:04.300
Typically, visualizing server usage reveals a bottleneck where your application consumes a large amount of memory without effectively utilizing CPU resources. The common practice in Rails for scaling is to deploy multiple instances on the server, which can quickly become expensive, especially under high loads.
00:03:30.949
To address this, we need to reduce the RAM footprint of our application so we can handle more requests while keeping Ruby busy with meaningful work. This involves optimizing memory use and improving IO performance.
00:04:02.680
One approach is to streamline your application code and remove unnecessary features. How many of you have worked on a project for more than a year? As the project evolves, legacy features may linger without being evaluated for their current utility, leading to unnecessary bloat.
00:04:51.970
To help with this, we've implemented profiling tools to check which aspects of our application are actually being utilized. Tools like Rubygems help us identify unused classes and methods, allowing us to streamline our code further. Additionally, we found that Ruby classes consume significant memory. For example, creating numerous short-lived classes can lead to inflated memory usage that isn’t reclaimed effectively.
00:05:30.600
To manage memory more efficiently, we should avoid singletons and monkey patches, particularly if they involve dynamically creating classes during runtime. This can lead to rapid memory consumption. We can also benefit from reusing regex patterns by assigning them to constants rather than creating new instances repeatedly.
00:06:03.600
Another best practice is to limit the gems used in your application. While Rails is fantastic for HTML and form building, it may not be the best choice for service-oriented architectures that primarily deal with JSON or XML. Instead, focus on lightweight gems specifically for JSON parsing to minimize memory consumption.
00:06:59.020
Moreover, consider utilizing alternatives to Rails for your web services, such as Sinatra or other frameworks that allow you to efficiently manage requests with lower overhead. For instance, by moving away from Active Record for certain parts of your application, we adopted the Sequel gem, which provides database interaction with reduced overhead.
00:08:02.480
While still allowing us to take advantage of the helpful features from Active Support, we’re able to minimize unnecessary dependencies and optimize our application further. By employing these strategies, we have observed significant improvements, including reducing our request processing time and increasing request per minute (RPM) capabilities.
00:09:10.460
When implementing multi-threaded applications in Ruby, it's important to note that Ruby does not offer true concurrency due to the global interpreter lock. However, you can achieve concurrency by utilizing IO operations effectively, as Ruby allows threads to be released while waiting for IO requests.
00:10:08.890
Particularly starting with Ruby 2.0, the IO performance improvements have enabled our server to handle more connections efficiently by spinning up multiple threads per process. Each thread can manage its own IO operations simultaneously, significantly improving request handling capacity.
00:10:57.920
Utilizing a well-structured thread pool management system is crucial, as it reduces thread creation overhead while maintaining responsive application performance. By carefully managing threads this way, we can scale our applications efficiently and effectively.
00:12:45.270
We also have to ensure that our application does not hang indefinitely while waiting for responses from backend services. It’s essential to establish limits for your timeouts and to implement connection retry logic without allowing one slow service to degrade the performance of the entire application.
00:14:55.000
When connecting to external services and databases, utilize connection pooling to limit the number of open connections, preventing overload of your backend systems. Additionally, avoid holding on to slow connections and instead consider terminating them if necessary to keep other requests active.
00:18:40.650
In terms of operational practices, maintaining proper logging and monitoring of your services is crucial to diagnose potential issues before they escalate. By being proactive about resource management and connection handling, we can create scalable and maintainable Ruby web services.
00:20:14.365
To summarize, we’ve discussed how to scale Ruby web applications through reduced memory usage, optimized IO operations, and using thread pooling effectively. By leveraging efficient practices and tools to streamline processes, we have successfully increased our application's request handling capabilities.
00:22:06.920
Our goal has been to reach a state where user experience is unhindered by the application’s performance. The strategies discussed in this talk can help any Ruby developer enhance their web services and scale them to new heights.
00:23:48.300
I appreciate your attention and hope that these insights will be useful as you continue to develop web services in Ruby. If you have any questions, please feel free to ask them now.
