Scaling Ruby with JRuby

Talks

Charles Oliver Nutter

@headius

#jruby

#java-virtual-machine-jvm

#database-performance

Scaling Ruby with JRuby

by Charles Oliver Nutter

In the talk, "Scaling Ruby with JRuby," presented by Charles Oliver Nutter at RubyConf TH 2022, the speaker delves into the capabilities and advantages of using JRuby, an implementation of Ruby that runs on the Java Virtual Machine (JVM). The primary aim of JRuby is to provide the Ruby development experience while leveraging the robust architecture and extensive libraries available in the JVM ecosystem. Key points covered in the talk include:

JRuby Overview: JRuby is an implementation of Ruby designed for the JVM and ensures the compatibility of pure Ruby code across environments. It focuses on maintaining the Rubyist experience while gaining benefits from Java libraries and performance features.
Version Updates: The latest release, JRuby 9.4, upgraded from Ruby 2.6 to 3.1, enhancing compatibility with modern Ruby features and libraries. Nutter emphasized the commitment to maintaining past versions while focusing on performance enhancements.
Performance Advantages: JRuby significantly improves scaling Ruby applications by using native threads, which allows for better resource utilization. Unlike traditional CRuby implementations, JRuby supports thread-based concurrency, enabling users to handle numerous requests concurrently without the overhead of multiple processes.
Getting Started: Setting up JRuby is straightforward, and developers can easily integrate it with existing Ruby on Rails applications. Nutter also provided tips for migrating existing Ruby applications to JRuby and highlighted tools like bundler and Active Record JDBC.
Benchmarking: Nutter showcased performance benchmarks that illustrated JRuby's capabilities compared to other Ruby implementations. He took a deep dive into memory usage, warm-up times, and concurrency performance, showcasing real-world application scenarios.
Case Studies: A notable success story was shared about a company that migrated a large Rails application from C Ruby to JRuby, leading to a significant reduction in operational costs and improved performance metrics, exemplifying JRuby's efficacy in production environments.
Future Prospects: The speaker concluded with a call for community involvement in testing JRuby 9.4, contributing to its development, and expressing excitement for future optimization opportunities and features akin to Ruby fibers for enhanced asynchronous capabilities.

In conclusion, Charles Oliver Nutter emphasized JRuby's valuable position in the Ruby ecosystem, providing developers with a powerful tool for building scalable applications while benefiting from JVM features. He encouraged the audience to explore JRuby for their Ruby applications and highlighted ongoing efforts to enhance its capabilities and community involvement.

00:00:00.120 foreign

00:00:15.120 Here we go. So, let me share my basic information. I've been programming as long as I can remember, but I've been doing it professionally since 1996, and I've been working with JRuby for 15 years now.

00:00:19.440 It's kind of amazing. Thank you to Red Hat, or IBM, for sponsoring our work for the past 10 years. It's amazing that they are willing to do this, and they even sponsor all of my trips. I'm not exactly sure why, but I'm not going to complain too much.

00:00:39.239 I was here for RubyConf Thailand in 2019. It was my first time here, and I was very excited to meet some JRuby users that I had never met before. There he is, in the back. I also got to try fresh durian for the first time. How many durian fans are here? How many people hate durian? It's almost the same number of people. I kind of dig it; I like weird stuff like that. I also got to see some Muay Thai when I went up to Chiang Mai. We paid a little extra for the VIP tickets, which allowed me to go up on stage—that was awesome.

00:01:16.860 Since then, during the pandemic, I learned to cook and watched a lot of streaming content, including a lot of Drag Race. I even got to meet Pangina Heels a couple of weeks ago when she was in town. If you're into any Drag Race stuff, come and talk to me. Natalia Plea Campbell should not have won.

00:01:30.299 I also want to say one more thing: thank you to Yarden. I got to speak at RubyConf even though I completely missed the CFP and didn't have a talk scheduled. She was going to speak about JRuby but couldn't make it, so I managed to take her slot. Make sure to give her a lot of love tomorrow; her talk is really cool, and she'll be here for the keynote in the morning.

00:02:00.720 Okay, let’s get to the topic at hand: JRuby. So what is JRuby? How many people are familiar with JRuby? Okay, probably about a third of the room. How many people are running something with JRuby? There’s a handful of them—good.

00:02:03.479 JRuby is simply put, an implementation of Ruby on top of the JVM. It is a Ruby implementation first. By this we mean we focus on the Ruby development experience above the Java side. We want Rubyists to feel at home when they start using JRuby. Of course, we get all the benefits from the JVM, which I will mention later. The general idea is that if it's just pure Ruby code, it should work. The vast majority of gems that you use in a typical Ruby application are just pure Ruby and should all work properly on JRuby. If something's wrong, let us know.

00:02:37.560 We don't have the same extension API, so gems that have a native library component sometimes need a replacement, or you can use any of the other JVM libraries to replace it. Also, there’s no forking because the JVM doesn't do that, and we have actual parallel threads. I'll show later.

00:03:09.900 Ruby compatibility status: we're very excited. Just before RubyConf last week, we released JRuby 9.4, which jumped from our old compatibility level of Ruby 2.6 up to Ruby 3.1. The vast majority of features from 2.7, 3.0, and 3.1 were shipped in 9.4.

00:03:14.760 We will continue to maintain the old release, JRuby 9.3 for at least the next year because some users won't move up to 9.4 right away. We always focus on compatibility before performance, so now is our opportunity to work on performance for a while before we have to implement Ruby 2 features. In our bug tracker, you'll find issues like this: there's a list for Ruby 2.7, 3.0, and a 3.1. We basically just make a list of all the issues or features that are in the Ruby news release file.

00:03:45.599 Anything that's a major feature added to those Ruby versions will be checked off on our checklist, and we go down the line to implement everything. Here, you can see one feature that we didn't quite get into 9.4 because we didn't think most people would care much about it. That’s one of the features that fell behind. We take the leftover check boxes and either throw them into bugs or create another list, so that we can easily know what features are still missing.

00:04:03.420 Of course, I mentioned Ruby 3.2 is coming. The release candidate just went out, I think. We will set up another checklist and, over the next few months, start getting those features implemented. There aren't too many big things that I've seen so far, but I think we can probably get this out in the next six months or so. It's nice to be caught up again.

00:04:48.660 So then, why do we want Ruby on the JVM in the first place? It’s a truly impressive runtime. It's widely supported, runs on every platform you can think of, has excellent JIT compiler support, and a myriad of different garbage collection options for various workloads and sizes. There are tens of thousands, if not hundreds of thousands, of libraries out there. If there's a feature you're looking for or a new library you're interested in, there's probably at least five of them for the JVM, and at least three of them are okay.

00:05:26.520 We also have a lot of tools for monitoring, profiling, etc. The idea behind JRuby is: write once, run anywhere. With JRuby, you can write a Ruby application and, with very few changes, move it over to another system or hardware platform, and it will run exactly the same. You don't have to recompile anything or make any changes.

00:05:53.820 I've shown this slide for years. This is one of my favorite tools, VisualVM. On the right side is the Visual GC plugin, which shows a live view of the garbage collector in the JVM. You can see younger generations fill up with garbage, then get cleared out as more objects are created. This example highlights the kind of tooling you get for free on the JVM. You can hook this up to a production app and monitor what's happening with memory and threads.

00:06:20.760 Speaking of threads, JRuby being on the JVM has real native parallel threads. This is just one Rails instance, one Rails worker running on JRuby, maxing out all the cores on my machine. It's very easy to do. You can take an entire site and run it with one JRuby instance, utilizing 10, 20, 100 threads for 500 concurrent users—no problem.

00:06:59.460 There's also a lot of fun stuff we can do on the JVM. This is an example from the Glimmer DSL framework, which is a GUI DSL for Ruby, and has multiple back ends. One back end is for the SWT component library. One of the examples on their site is a little Tetris game implemented in JRuby with SWT, demonstrating the kind of cool applications we can create.

00:07:10.080 As for users, many cool companies have utilized JRuby over the years. Some of these companies have moved on. NASA, for example, had a JRuby application that coordinated some radio telescopes in the search for extraterrestrial life. Though they used C++ for the actual telescope operations, the synchronization was done via a JRuby app. I also know that the Oslo International Airport uses a JRuby-based Android refueling terminal for airplanes, which is a bit terrifying to think about, but I’ve flown through it multiple times and never had any problems.

00:07:47.520 Getting started with JRuby is easy. Of course, you can visit our website, download it, or you can use a Ruby installer like RVM. All you need is a JVM or JDK installed on your system. Then, just install JRuby through RVM or any Ruby install method, and you’ll be up and running. We also provide the tarballs and a Windows installer for those that use it.

00:08:00.540 Here is a little IRB session with JRuby. We started it up and are actually calling into core JVM libraries as if they were regular Ruby libraries. Here I get the processor count and free memory on the system. Anything available on the JVM as a library can be pulled into a Ruby application and utilized just like a standard Ruby class.

00:08:43.920 Now, let's discuss scaling. JRuby on Rails indeed runs well. Since about 2007, we've managed to get Rails working on JRuby—back then, it was Rails 0.12. There are numerous JRuby on Rails applications out there. I don’t even know how many hundreds or thousands exist, but I hear about new ones every time I attend an event. I believe it's one of the best ways to scale a Ruby application. You can utilize one JRuby process to max out all the system's cores while taking advantage of your hardware’s parallelization capabilities. Generally, this results in using less memory than having multiple worker processes to run your site.

00:09:53.640 So, if you're just starting with JRuby on Rails, the process is straightforward. Rails understands JRuby and will generate an app using our version of the database libraries, configured appropriately for JRuby on Puma and Salon. There are a couple of gotchas in Rails 7 that we are working through. If you encounter those, please let me know. The primary adjustment is switching from workers to using threads. If you were using 10 or 20 worker processes for Rails before, replace them with an equivalent number of threads to saturate the system. A ratio of about 2N plus one is advised since you'll have threads waiting for database responses or I/O operations. Therefore, having approximately twice as many threads as cores in the system should allow you to manage hundreds of thousands of concurrent users effortlessly.

00:10:57.300 For existing apps, the simplest way to start is to try bundling. You might encounter libraries with extensions that are unsupported on JRuby; these can be replaced with pure Ruby libraries or other JVM libraries. However, it may equally just run right out of the box. If you manage to get your new app up and running, you'll see it functioning as intended with Rails 7 on JRuby, and you’ll have access to all the information about the JVM you’re utilizing.

00:11:37.380 So yes, Rails 7 does work and is mostly functional. We accomplished this just before RubyConf. We're still ironing out some kinks, particularly within Active Record since it has a considerable amount of code in the Rails Active Record database drivers that we need to replicate for the JVM equivalent, the Java Database Connectivity libraries. Performance and compatibility look good, and I will present some performance results later. As I mentioned, there are a few gotchas, but we're addressing them and will support you through it.

00:12:05.160 The testing process for JRuby is relatively solid across most areas except for Active Record. Most of the minor failures revolve around float comparisons, which can vary slightly or dates presenting slightly different microsecond values. However, these are not critical issues—just manifestations of platform differences since the JVM handles dates or numbers a bit differently, but it usually doesn't impact user experience.

00:12:44.400 Our database adapter uses Active Record JDBC, wrapping the standard database libraries for the JVM, aligning them with the Rails API to maintain compatibility. We adopt a numbering convention where, for example, Rails 6 corresponds to JDBC driver version 60.0 and version 70 for Rails 7, and so on. SQLite and MySQL are working well enough for me to conduct all the benchmarking and testing for this talk.

00:13:10.740 There are some minor failures we need to sort out, but nothing significant. PostgreSQL, however, involves a lot more code, and it’s taking longer to finalize those implementations.

00:13:24.900 Now, about scaling: if you're talking performance and scaling benchmarks, consider what's significant for you as a developer. I can display benchmarks all day long, but unless it's relevant to your company or application, it’s not beneficial. Is it straight-line performance? Are you working with massive data churn? Do you require high concurrency to cater to a large number of users? Perhaps you need extremely fast startup time for command-line utilities. In that case, JRuby might not be ideal. Similarly, if you require frequent deployment or have limited memory constraints, you might face challenges with JRuby.

00:13:52.560 It's crucial to test your code under real conditions. Today, I have a couple of situational benchmarks, which I hope are representative of what you may be currently using. However, keep in mind that what appears beneficial in a microbenchmark may not reflect what happens in production. Various Ruby implementations have claimed remarkable benchmarks for numerical algorithms or method calls within loops, which have not always translated into real-world performance.

00:14:50.940 Two example benchmarks will be shown: the little Rails blog benchmark, which has been used by the YGIT developers mostly to identify bottlenecks within the Rails framework itself, and a full end-to-end Rails blog running on MySQL, which is perhaps more akin to an application that you might deploy.

00:15:11.760 The Rails benchmark consists of a simple scaffolded blog application running on a SQLite database, utilizing a single-threaded all-in-one process. It simply runs a loop, cranking through a set of requests as quickly as possible—this isn't particularly representative of a production application, but it's useful for testing request handling and a bit of database interaction.

00:15:43.080 I’ll be benchmarking Ruby 3.1.2, both with and without YJIT. YJIT is starting to show benefits. I’m also including Truffle Ruby, though it has been inconsistent in terms of performance, especially with Rails applications. My numbers vary significantly, with smaller scripts performing well, but rail implementations not matching expectations yet.

00:16:19.860 For JRuby, I’m running version 9.4 on Java 17, which is the latest long-term supported version and offers a substantial performance boost compared to earlier versions. After warming the applications, I ran 2000 requests. In my tests, Ruby 3.1 was able to handle 2000 requests in a little under a millisecond.

00:17:08.940 YJIT is producing impressive results over previous implementations. For my system, Truffle Ruby was slightly faster than YJIT, while JRuby was even faster still, surpassing all three options. Admittedly, this benchmarking is somewhat contrived and does not encapsulate real-world conditions comprehensively.

00:17:52.740 The takeaway from these benchmarks primarily reflects straight-line performance. While there's some database access involved, the bulk of these tests are on the request pipeline. What we’re missing here are concurrency, startup time, warm-up issues, and memory usage—all crucial factors to consider.

00:18:04.980 As for memory footprint, complex runtimes like JRuby and Truffle Ruby tend to use considerably more memory. We’ll have multiple copies of code in memory, generate native code with a JIT compiler, and utilize garbage collectors that require additional space. This behavior is something we have to manage and mitigate.

00:18:31.680 However, there are key distinctions with C Ruby, which has been optimized for startup time and memory. It can run effectively in low-memory environments, significantly quicker than JRuby. We're not trying to outperform C Ruby in those aspects, recognizing that JRuby makes sense for larger applications.

00:19:05.700 In the context of this benchmarking for JRuby, the YJIT doesn’t notably affect memory use. During tests, C Ruby utilized about 80 megabytes, while Truffle Ruby occupied around 2.4 gigabytes. JRuby, on the other hand, typically stayed around 900 megabytes. The JVM often conspires to use significant amounts of memory if not directed otherwise.

00:19:37.920 Another critical aspect of performance to highlight is warm-up time. Optimizing runtimes like JRuby often take longer to reach operational states due to the extensive code interpretation to compile. While the performance of C Ruby peaks quickly, JRuby takes its time to see the effects. Conducting approximately six iterations of 2,000 requests enables it to stabilize performance, but it needs to warm up sufficiently before reaching peak operational levels.

00:20:40.200 Consequently, expect that as you run JRuby applications over more extended periods, they will progressively approach and ultimately exceed the performance of C Ruby. This approach necessitates patience during the initial phase of executing applications. If your application requires immediate responsiveness, consider pre-warming that application by executing a script that generates requests prior to production launch, ensuring it's ready to serve major requests.

00:21:05.500 The JRuby architecture implies that the Ruby code necessitates parsing into Ruby instructions that are subsequently compiled into an intermediate representation. The associated parser and compiler are initially executed by the JVM. Thus, the startup and warm-up impacts result from these operations.

00:21:33.300 Now, as performance improves, the longer an application operates, the expectation is to reach peak efficiency before system shutdown or underutilization, accommodating the necessary warm-up. In comparing warm-up time, we find that JRuby will ultimately execute at faster speeds than C Ruby after sufficient runtime. Truffle Ruby demonstrates an even more extended initialization curve before attaining optimum performance.

00:22:21.600 Benchmarks reiterate that the most relevant results are derived from testing your own code, which is why I encourage everyone to experiment with JRuby, Truffle Ruby, and C Ruby alongside YJIT to truly ascertain how your applications perform.

00:23:07.200 To offer a clearer picture, we will conduct a full end-to-end test using a Puma MySQL Rails application, employing Siege to maximize concurrency and assess the speed of request handling, aiming to expose performance nuances and further insights.

00:23:52.020 When it comes to scaling Rails, this has often been a classic challenge for C Ruby due to the absence of concurrent threads. To deal with performance, developers typically have to deploy numerous worker processes. Despite clever copy-on-write techniques, the overhead for memory utilization remains as each process maintains individual garbage collectors and JITs, compounding the workload.

00:24:30.480 Conversely, when running Rails on JRuby, you have the advantage of threading. You can maintain a singular process, maximizing your resources with multiple threads that can serve extensive user loads while simultaneously reducing overhead. The example I’m presenting comes from a typical Rails application using MySQL, running on my i7 work machine.

00:25:17.880 In this test, I ran C Ruby with 16 worker processes, each comprising two threads, effectively utilizing 32 threads overall. Then, with JRuby, I aimed for 32 concurrent threads total. Though I experimented with Truffle Ruby, it encountered many errors.

00:25:53.640 For benchmarking, I used my work machine, though for scientific testing, ideally, you'd have separate machines for drivers and appropriately set up database servers. Nevertheless, this suffices for demonstrating the primary functions, and I can exhibit the warm-up metrics from this application.

00:26:24.540 We are measuring requests-per-second rates, with a higher number indicating better performance. In this context, we observed C Ruby processing around 2,000 requests per second against a blog post entry in the scaffolded blog application. Additionally, we noted a performance boost from YJIT, which improved requests per second by approximately 300 or more.

00:26:55.020 While Truffle Ruby didn’t meet expectations, I spoke with the team, and we acknowledged issues with concurrency tied to the necessity of locking C extension calls. As a result, we cannot utilize concurrency under this model unless we disable that lock, presenting survival risks for stability.

00:27:39.540 Through the JRuby tests, we see an easy 2x improvement over C Ruby with YJIT, despite not having engaged in any performance optimization in years. So we are primarily focused on compatibility developments.

00:28:07.080 Regarding memory consumption, JRuby exhibited favorable results relative to C Ruby. Running the Rails application on my machine, JRuby handled nearly 1.4 gigabytes of memory, indicating room for optimizations down into a comfortable 900 to 1,000 megabyte usage spectrum.

00:28:23.640 Finally, the warm-up period for JRuby is not excessively long for this application, but it generally requires around 10 seconds with several thousand requests to ensure functionality. By executing representative scripts, you can expedite your application’s readiness for production to meet your operational needs.

00:28:51.600 In conclusion, if you aren't sold already, I've got a success story—there was a company that had significant scaling issues caused by C Ruby. They maintained a large Rails application on 40 extra-large instances, which is exceptionally costly for production computing. Their system averaged 100,000 to 150 requests per minute, though with response times of 50 to 75 milliseconds, it incurred huge operational costs.

00:29:44.760 The organization dedicated two weeks for a proof of concept to migrate the application to JRuby on Rails. This pivot allowed them to enhance threading while utilizing fewer processes. With modern JVMs, they managed to scale down their needs to just 10 extra-large instances, achieving a 75% reduction in production deployment expenses.

00:30:13.800 Furthermore, they experienced consistently over 150,000 requests per minute with better response times, underscoring the effectiveness of leveraging JRuby. These types of transitions demonstrate that JRuby successfully applies to production while delivering significant financial savings. As for future directions for JRuby, we’ve released version 9.4, and we'd love for you to test it out.

00:30:59.340 Join us by filing issues and contributing pull requests with features that may be missing or improvements on Ruby. We're continuously working on updates for Rails 7 while tackling various features and the PostgreSQL driver adapter.

00:31:19.920 I'm particularly excited about returning to optimization work over the following months as I focus on improving JRuby's capacity. Additionally, we are investigating new JVM features, including an exciting concept similar to Ruby fibers- allowing us to utilize native threads without relying on the older implementations. This advancement will significantly enhance asynchronous application performances, ultimately leading to many new capabilities and libraries accessible to JRuby users.

00:31:58.860 Thank you for your time!

RubyConf TH 2022