YJIT Makes Rails 1.7x Faster

Talks

Takashi Kokubun

@k0kubun

#yjit-yet-another-ruby-jit

YJIT Makes Rails 1.7x Faster

by Takashi Kokubun

The video presented by Takashi Kokubun at RubyKaigi 2024 revolves around YJIT (Yet another Just-In-Time compiler), focusing on how it enhances the performance of Ruby on Rails applications, boasting improvements of up to 1.8 times faster execution. Key points discussed include:

Introduction to YJIT: Takashi explains that YJIT is a JIT compiler that can be enabled in Ruby applications, designed to significantly speed up method execution by converting frequently-called methods into native code.
Production Use Cases: The speaker shares a success story involving Shiseido's ST renderer service, which achieved a 177% performance boost in a production environment with high traffic, demonstrating the effectiveness of YJIT.
Enabling YJIT: Various methods for enabling YJIT in production are outlined. This includes adding the '-W' option to the Ruby command, utilizing environment variables, and incorporating built-in functionality from Ruby 3.3 and Rails 7.2.
Memory Management: Takashi emphasizes the trade-off between performance gains and memory usage, suggesting adjustments to 'YJITEXECMSIZE' to optimize memory allocation without compromising on speed.
Ruby 3.3 Enhancements: Improvements in Ruby 3.3 that contribute to YJIT's performance include a higher optimization ratio for method calls and enhancements in exception handling and stack value management.
Further Optimizations: The speaker notes that ongoing work for Ruby 3.4 will further refine these optimizations, including better handling of method inlining for standard operations in Rails applications.
Final Recommendations: The session concludes with a strong recommendation for developers to upgrade to Ruby 3.3 and enable YJIT in their applications to leverage the observed performance improvements, encouraging community contributions for better optimization.

00:00:10.759 Hi, everyone! Today I'm going to talk about a position called YJIT that makes Rails 1.7 times faster.

00:00:13.440 I need to apologize; this was a lie. As of this morning, it's actually 1.8 times faster. My name is Takashi Kokubun, and I work for Shield on the Ruby Infrastructure Team.

00:00:23.160 Shield has a Ruby infrastructure team that was explained in the previous session. It consists of five people. Under that team, we have various subteams such as the Ruby Infra Team and the Ruby DX Team. I'm part of the Ruby Infra Team, which consists of those people.

00:00:42.520 Now, I'm going to talk about YJIT. YJIT is a Just-In-Time (JIT) compiler. The purpose of this talk is to make you feel that you need to use it in production. You should also consider upgrading your Ruby version to 3.3, the latest version.

00:01:01.760 What it does is when YJIT is enabled—it's disabled by default—you need to turn it on explicitly. If you enable YJIT and call some methods more than 30 times, it triggers JIT compilation, which converts that method to native code. From that entry point, the method's code will also be compiled.

00:01:39.320 In production, we are using YJIT for our most important services. One of those is Shiseido's ST renderer, which is 177% faster in production. This service has the largest traffic in our company—billions of requests—and is battle-tested. I've talked to many people here at RubyKaigi, and they've told me that they are using YJIT and thank us for it.

00:02:25.200 Now, I wonder how many of you are currently enabling YJIT in production? Could you raise your hands? I want everyone to raise their hands next year, so please consider enabling it.

00:03:07.879 Enabling YJIT can be done in several ways. The most basic method is to add the '-W' option to the Ruby command. In addition, the Ruby opt environment variable is available by default. There's also a special environment variable specific to YJIT: RUBY_YJIT_ENABLE. If you pass this environment variable, YJIT will be turned on by that configuration.

00:03:38.120 Starting from Ruby 3.3, we have a Ruby VM method called 'rb_vm_yjit_enable' that can be called from a Ruby script without needing to set any configuration beforehand. The good news is that Rails 7.2 will include a default initializer that enables YJIT when using Ruby 3.3. As long as you upgrade Ruby and Rails to these versions once they are released, YJIT will effectively become the default JIT compiler in production.

00:04:21.560 Please use it when you upgrade Rails to 7.2, or you can also use it as long as you upgrade to Ruby 3.2 or 3.3. One of the challenges in enabling YJIT in production is memory usage. There is always a trade-off; gaining performance might mean using more memory.

00:05:00.760 One trick we're recommending to the community is to adjust the parameter 'YJIT_EXEC_MSIZE,' which controls the number of bytes allocated for JIT code. By default, this is set to 48MB as of Ruby 3.3. As of Ruby 3.2, it was 64MB. This default is meant to maximize speed, but in many deployments, you may want to reduce the memory usage for your service instead.

00:05:50.280 In that case, you could try reducing it to 32MB, 24MB, or even 16MB. I have heard that some teams are using a smaller memory configuration, and I think as long as your application is smaller than theirs, you should be able to do that too. The caveat is that 'YJIT_EXEC_MSIZE' only controls the JIT code size, and YJIT also needs to maintain metadata for each JIT code.

00:06:39.680 If you configure 'YJIT_EXEC_MSIZE' to, say, 32MB, it could also require additional metadata memory, such as 64MB or 96MB. Managing this configuration helps control YJIT's memory usage. We provide tips for production deployments in the Ruby documentation for YJIT, so please check it out.

00:07:01.440 The next part is about improvements made in Ruby 3.3 that help explain why YJIT is getting faster with each version upgrade. I want you to understand these upgrades and encourage you to use the latest version in production.

00:07:55.360 Among the many features released in 3.3, one significant change improved Rails performance by 7%. Although there was a previous discussion of 5% improvement in string manipulation, 5% is significant—especially if you've worked on performance optimization. In this context, a 7% improvement is even more substantial.

00:08:39.240 If we look at the list of changes in Ruby 3.3, can you guess which one was the most significant in terms of Rails' performance? Someone from the audience suggested that it was the implementation of the blank or present method. While interesting, that was not the case. The most significant change is actually more complex and can be challenging to explain.

00:09:44.440 Let me explain a method called 'fullbacks'. In the Ruby virtual machine, we compile Ruby source code into instructions called 'instruction sequences'. Although you don't need to understand all the sequences, I wanted to show code to illustrate what is happening behind the scenes. During compilation, we optimize the instructions as long as we can successfully compile each instruction.

00:10:08.919 In many cases, all instructions are supported by YJIT, yielding a 'ratio in YJIT' of 100%. However, some Ruby code patterns are not optimizable by YJIT. For instance, certain keyword rest patterns do not support optimization. When we encounter unsupported patterns, we need to generate a side exit. A side exit means exiting the JIT code and returning to the interpreter for execution, which is slower compared to executing specialized machine code.

00:11:00.480 In this context, we might report a 'ratio in YJIT' of only 33%, suggesting that only one-third of the code is optimized. We've been able to compile most instructions, but complications arise when unsupported method codes are encountered.

00:11:54.000 What we did in version 3.3 was to adjust this. By leveraging the functionality of previous JIT implementations, we updated how we handle these scenarios. For example, by calling back to the interpreter correctly, we can optimize the instructions following a method call. This increased the optimization ratio significantly.

00:12:43.440 In Ruby 3.2, the optimization ratio typically dropped to about 90%, meaning 10% of Ruby virtual machine instructions were not compiled. However, upgrading to Ruby 3.3 improved that ratio to an impressive 99%.

00:13:48.480 Please upgrade Ruby to 3.3. This is a main takeaway from this presentation. We've also discussed unsupported codes and the concept of 'Megamorphic' dispatch.

00:14:32.960 Megamorphic dispatch is a significant version of polymorphic dispatch. For example, when a method calls an 'X' method through various sub-classes, the dispatching mechanism needs to be efficient. Ruby's JIT can cache multiple entries in a single method call, optimizing performance. However, there is a limit to how many chains we can append before the performance suffers due to overhead.

00:15:22.760 In previous versions, when a side exit occurred from this method, the instructions that followed were not compatible due to this limitation. But in Ruby 3.3, we've optimized how we handle this to improve performance drastically. We can reduce exits and enhance the efficiency of method calls.

00:16:34.960 The next topic concerns exception handling. Exceptions in Ruby happen at various points, not only in the 'raise' method. Internally, almost any interruption, like a 'break' in an EACH block, is managed by an exception mechanism in the Ruby VM.

00:17:25.680 When you reach a 'break' statement, Ruby needs to pop multiple frames off the stack. The mechanism currently allows easy handling of this in Ruby 3.3, which greatly improves performance by providing JIT compilation support for managing exceptions.

00:18:17.919 As a result, the exception handling ratio has moved from about 60% in Ruby 3.2 to 100% in Ruby 3.3.

00:19:15.440 Though this upgrade does improve the performance, keep in mind that the increase in code complexity can potentially lead to greater memory consumption. However, our experience suggests that fine-tuning the memory settings should not significantly hinder performance.

00:20:14.560 The next topic is new behavior for stack values. The Ruby virtual machine is a stack-based VM, and we have some optimizations for accessing stack values, streamlining operations!

00:21:25.440 In version 3.2, operations on Ruby code primarily utilized memory, which is generally slower than accessing registers closer to CPU. Ruby 3.3 will leverage this by optimizing how values are managed during computation.

00:22:11.360 Additionally, the team is working on further enhancements for Ruby 3.4, including optimizations around lower variable access, expected to provide significant performance improvements.

00:22:59.360 Improvements in method inlining have also been made, particularly for Ruby on Rails applications. For instance, Rails 7.2 introduces enhancements for present and blank methods, providing faster performance compared to earlier versions.

00:23:53.520 Ruby 3.3 allows inlining of single-line methods that return immediate values like integers or true/false values. Such optimizations lead to significant reductions in the instruction overhead for common Ruby operations.

00:24:54.360 All these changes continuously enhance the core Ruby ecosystem, allowing for cleaner Ruby code and improving overall performance across the board. This includes techniques for optimizing method calls in C code, where we acknowledge existing challenges but strive for effective implementations.

00:25:54.560 As we innovate to further improve performance, we encourage contributions to the Ruby C implementation to replace C methods with Ruby, fostering clarity and maintainability.

00:27:04.400 Finally, we must also stress the importance of enabling YJIT in production and upgrading to Ruby 3.3. Many users observe measurable performance gains, so please consider upgrading your applications to the newer versions as they become available.

00:28:10.720 That's all! Thank you!

See Slides on speakerdeck.com

RubyKaigi 2024