Talks
Improving CVAR performance in Ruby 3.1
Summarized using AI
Improving CVAR performance in Ruby 3.1
by Eileen M. UchitelleThe video presents a talk by Eileen Uchitelle at RubyConf 2021, focusing on the performance improvements of class variables (CVARs) in Ruby 3.1. The discussion explores the complexities of how CVARs operate, particularly in terms of inheritance. Key points include:
Understanding Class Variables: CVARs are defined at the class level, indicated by two '@' signs, and have distinct behaviors when it comes to inheritance compared to instance variables. They are shared across class descendants, which can lead to unexpected results when setting them in subclasses.
Performance Issues: The complexity in CVAR inheritance causes significant performance slowdowns, especially as the inheritance chain deepens. Benchmarks indicated that accessing CVARs could be much slower with deep inheritance.
Introducing a Cache: Uchitelle and Aaron Patterson developed a caching mechanism that minimizes the performance hit caused by CVAR reads and writes. This cache helps reduce the frequency of walking the entire inheritance chain to access class variables, thus improving performance.
Real-World Impact: Their improvements resulted in a six to seven percent increase in request handling in Rails applications, demonstrating that the caching solution offers substantial benefits in practical scenarios.
Concerns and Negotiations with the Ruby Core Team: Uchitelle discussed the skepticism faced from Ruby's maintainers regarding increased complexity and potential overuse of class variables due to performance enhancements. They emphasized the importance of understanding trade-offs in open-source contributions while effectively advocating for their changes. Ultimately, the cache was merged and rolled out to users with Ruby 3.1, highlighting the collaborative effort and negotiation skills required in open-source development.
The talk concludes with a call to action for developers to contribute to Ruby, emphasizing that collective improvements can ensure Ruby's longevity and efficiency as a programming language.
00:00:10.160
Hi everybody!
00:00:11.759
I’m so happy to be back in person this year at RubyConf, finally, after two years of not seeing any of your faces, except for on screens.
00:00:16.880
Before we get started, I want to say thank you to all of the organizers at Ruby Central, the volunteers, and everybody from Confreaks who are helping to ensure that this conference happens and is live-streamed. After doing several remote talks over the past couple of years, I can appreciate first-hand how much work goes into audio, lighting, recording, and live streaming. I'm very glad to not be worrying about my Wi-Fi going out while giving this talk.
00:00:34.000
I’d also like to thank all of you in the audience for attending RubyConf in person this year, and everyone watching on the live stream at home. I hope that you’re all enjoying this conference as much as I am.
00:00:58.800
For those who haven't met me before, I’m Eileen Uchitelle, and you can find me online at the handle Eileen Codes. I have been a member of the Rails Core Team since 2017. The core team for Rails is similar to the Ruby Core Team in that we work to drive the future of the framework.
00:01:07.119
We plan releases, merge contributions, and build functionality to improve the framework in order to support applications as they grow. I work at GitHub, where I am a Principal Software Engineer focused on improving the Rails framework and Ruby language so that we can continue to use them for the long haul. Our team at GitHub works to find areas where we can improve performance, build functionality, fix bugs, and push the boundaries of Rails and Ruby.
00:01:31.920
In the past year, we’ve made many improvements to Ruby, including making the superclass method faster, fixing bugs, updating gems to support Ruby 3, sending pull requests to Shopify’s YJIT, and of course, what we are here to talk about today: improving the performance of class variables.
00:01:58.479
Earlier this year, I worked with Aaron Patterson from Shopify on adding a cache to class variables to make them faster. On the surface, adding a cache probably sounds like a relatively simple project, but it took us about six months — or maybe even longer — and a few different attempts to accomplish this. It turns out that class variables in Ruby are incredibly complex.
00:02:25.760
So today we’re going to talk about how class variables work, what makes them complex, why they were slow, and how we made a cache to improve their performance. After diving into the technical details, we’ll take a look at what we learned, how to handle trade-offs in open source, and lastly, why it’s so important that we all work to make improvements to Ruby.
00:02:39.760
Before we dive into Ruby internals and how we improve the performance of class variables, let’s first look at what a class variable is and how it works. A class variable, or '@@cvar', is a type of Ruby variable that is defined on the class level rather than the instance level.
00:02:57.680
You're probably very familiar with instance variables, but you’re less likely to have worked directly with class variables. They’re a bit different and can be difficult to reason about; you’ll often be told to avoid using them. Literally, every Ruby book tells you to avoid them. Class variables are denoted by two '@' signs instead of one.
00:03:19.599
In this class, we are setting a class variable called '@@cvar', and we have a method to read that class variable. That all sounds pretty simple, right? But what could be so complex about class variables based on this example? Well, class variables become a lot more complex once you look at how inheritance works.
00:03:35.039
To understand the complexity, let’s first take a look at class instance variables and how they work with inheritance. Here we have two classes: 'Dog' and 'Puppy.' Dog has an initializer with the name instance variable set to 'Arya' and an owner instance variable with the value 'Eileen.' I do have a dog named Arya, so this is real.
00:03:49.280
Puppy inherits from Dog and sets the value of the name to 'Sansa' but does not set the owner variable. When we read these variables in Ruby, we can see that the @names are isolated to their instance. While Puppy inherits from Dog, it doesn't automatically inherit initialized instance variables from Dog.
00:04:00.640
We can see here that the name on Dog is set to 'Arya' and owner is set to 'Eileen'. The name on the Puppy instance is set to 'Sansa' and owner returns nil. Instance variables are bound to the current self, which in this case is the instance of the class Dog or Puppy. The instance variables on Dog are isolated from instance variables on Puppy. Initializing a Dog instance doesn't change Puppy, and initializing a Puppy instance doesn't change Dog.
00:04:41.840
If you’ve been writing Ruby for a bit, you’re likely already familiar with how this works, and it isn’t surprising. The inheritance model is pretty straightforward when it comes to instance variables. Class variables, on the other hand, behave very differently regarding inheritance, which can cause confusion around how to use them properly.
00:05:04.800
Let’s take a similar example but with class variables. In this example, Dog has a class variable set to '@@name' with the value ‘Arya’, and an owner class variable set to 'Eileen.' Puppy inherits from Dog and sets the @@name to 'Sansa.' Just like the previous one, let’s see what happens when we read these class variables.
00:05:31.519
Here we can see that Dog's owner is 'Eileen' and Puppy's owner is also 'Eileen.' Remember, instance variable of Puppy returned nil, so that’s already different. But let’s see what the name is on Dog. Wait, Dog name is set to 'Sansa?' Shouldn’t it be 'Arya'? And Puppy’s name is also set to 'Sansa'. How can that be? Does this look like a bug?
00:05:55.920
Well, it might look like a bug, but this is actually how class variables are supposed to work. Class variables are shared among their descendants, so think of class variables as global; they can only have one value for a class in the same inheritance chain, and the last one to set the class variable wins. Every time you write or read a class variable, Ruby will walk the entire inheritance chain, all the way up to BasicObject, looking for the top-most class in the chain that sets the same class variable.
00:06:29.199
So, if 'name' is set on both Puppy and Dog, the actual value of 'name' is stored on Dog, which is the parent class of Puppy. Dog will also store the value of 'owner.' Then, when Puppy sets the name to 'Sansa,' Arya will get overwritten as the value stored on Dog because the last class in the chain to set the class variable wins. Even though the value is stored on the Dog class, that’s a mouthful.
00:06:51.680
The information for the class variable value is stored in a hash table, which is accessed from the Dog class since Dog is the topmost class that also has a setter for 'name.' Fun fact: class variables and instance variables are stored in the same hash table structure in Ruby. This is possible because the double '@' sign is not a valid instance variable name, so instance variables and class variables will never clash in that table, allowing them to share the same space.
00:07:24.240
You might be looking at this and thinking, 'If the hash table is stored on the topmost class, why don’t we just stop at Dog? Why does Ruby have to keep going all the way up the chain to BasicObject every time a class variable is written or read?' Well, Ruby needs to do this to handle what's called class variable overtaken. Since you can reopen classes in Ruby or mix in modules, Ruby cannot be sure that there isn’t already a class variable with the same key set elsewhere.
00:07:46.800
So, every time a class is referenced, Ruby checks through the entire chain for an existing hash table, and if it finds one, it raises an error. Let’s look at an example. Here we have the Dog class defined without a class variable, and the Puppy class defines a name. In this case, the class variable hash table is going to be stored on Puppy, not Dog, because Dog did not set a name class variable.
00:08:03.680
We then reopen the Dog class and define a name class variable in that class. When our code tries to read the name class variable on Puppy, it's going to raise an error: 'Class variable name of Puppy overtaken by Dog.' What’s happening here is that when the name class variable is set on Dog, the hash table that was stored on Puppy gets overtaken because the class was reopened.
00:08:31.360
The class variable needs to be stored on the highest class in the chain that has that class variable. So, when Puppy name is called, the hash table that was stored there essentially gets broken because we reopened the Dog class and set the name class variable higher up. As you can see, class variable inheritance is incredibly complex, which is one of the reasons it’s such a disliked feature by Rubyists.
00:08:56.800
Now that you understand how inheritance and overtaken work, you’re probably not surprised to find out that the overtaken check means that class variables can be pretty slow. In fact, the deeper the inheritance chain, the slower it will be to write and read class variables. If you have 100 classes in your inheritance chain, Ruby is going to need to traverse all 100 classes, plus those leading up to BasicObject, for every single class variable you write or read.
00:09:08.560
So, let’s take a look at some benchmarks to see how slow it really is. Here we have a script that generates a list of letters that we can use for module names. Class one here includes one module, while our second class includes 30 modules, and a third class includes 100 modules. In this example, we’re using modules instead of classes, but the inheritance behavior is exactly the same.
00:09:28.720
Then our benchmark reads the class variable getter from each of our classes to demonstrate that when the inheritance chain is deeper, class variables get slower. Who wants to see the numbers? In Ruby 3.1, at the time, including 30 modules with the class variable was 2.8 times slower than including one module, and including 100 modules was 8.5 times slower than including one module. That’s a pretty significant difference.
00:10:00.640
It’s clear that you don’t have to worry about class variable performance if you only have one class in your inheritance chain. But if you have 100 modules or classes, the class variable behavior is very likely affecting your application’s performance. Okay, but how often do we actually have a deep inheritance chain that uses class variables?
00:10:28.640
When working on improving performance, it’s important to realize that a microbenchmark like this that shows an improvement might not actually translate into meaningful performance improvements in your application or library. So once we knew how slow class variables were, the question became how often are class variables used?
00:10:49.760
When they are used, how often is the inheritance chain deep enough to warrant fixing the performance of class variables? It’s not going to be worth fixing performance if no one ever uses them. So while you may not use class variables often in your own code because you’re told not to, you might be surprised to find out that Rails uses class variables extensively for application configuration.
00:11:01.760
One of the most well-known variables among class variables is probably ActiveRecord-based configurations. This class variable stores all of your database configurations for your Rails application. Additionally, every time you see 'Matters Accessory' in Rails, it’s actually a feature of Rails that creates a class variable. It’s a DSL that you can’t really see, but if you go deep into the code, you can see the class variables.
00:11:31.760
So it wouldn’t be a big deal that Rails uses class variables if the inheritance chains were super short, right? But when you look at the ancestors of a few key classes, we can see that Rails' inheritance chains are often quite deep. In a vanilla Rails application, ActiveRecord has 73 ancestors, ActionController has 71, and ActiveJob has 29. All three of these use class variables, and I didn’t check the other libraries, so it might be even worse than that.
00:11:59.440
This means that if you’re using Rails, the performance of class variables is likely affecting your application’s performance in a real way. In order to address this, Aaron Patterson and I set out to build a cache for class variables.
00:12:18.320
When I first started working on this, we didn’t realize how complex class variables were — at least I didn’t. It took us a really long time to figure out exactly how the code worked and where all of the issues with overtaken were. What we wanted to pursue, however, was to make class variables faster, because we knew from experience that their slowness was affecting every single application.
00:12:49.920
If we could make class variables faster, we’d make Ruby faster. If we make Ruby faster, we make Rails faster, and if we make Rails faster, we make our applications faster — then your applications are faster too. This was an opportunity to improve the performance of every single codebase that uses class variables.
00:13:24.240
So, as we saw earlier, class variables are stored on a hash table on the highest class that sets the variable. It made sense to store the hash table for the cache next to the class variable hash table. Then we could rely on the existing class variable code to handle creating and busting the cache alongside it.
00:13:59.520
The cache hash table holds the class variable name, pointing to an inline cache. The inline cache is a pointer back to Dog and tracks a value called ‘global cvar state.’ The cache is stored on the topmost class, so when Puppy needs to find its class variable, the cache will be created by Dog the first time we read or write a class variable.
00:14:35.520
Ruby will walk all the way up the inheritance chain looking for the topmost class that has a setter for the name class variable. While doing this, an inline cache is created and stored in the bytecode, so that Puppy can access it when needed.
00:14:57.280
The next time we rewrite the class variable, Ruby will use the cache for the lookup instead of walking up the chain. For example, when we call ‘Puppy.name’ in an application, Ruby will find the ‘global cvar state’ from the cache and compare it to ‘global cvar state' value. If they match, that means we’ve seen this class variable before, and our cache is valid.
00:15:23.680
The cache then returns the Dog class, which informs Ruby where to look up the class variable hash table, eliminating the need to traverse the inheritance chain again. We know exactly where the hash table is stored, and then the value 'Sansa' will be returned from the Dog class's ivar hash table. This allows Ruby to skip walking the inheritance chain for any subsequent calls once the class has been created.
00:15:47.520
In cases of class variable overtaken, the global cvar state will have a different value for the getter global cvar state, and the cache is ignored. If those values differ, Ruby will walk the ancestor chain to build a new cache for the next time. That’s basically it. The overall cache ended up having a relatively simple design.
00:16:19.280
While the cache ended up with a simple design, it took us a long time to get a good handle on how class variables work in Ruby. The majority of our time was spent tracing the code. Every time we would pair again, we would ask ourselves how they worked, and we had to write it down eventually.
00:16:53.200
When we were done adding the cache, it was time to run benchmarks to see if our cache was indeed faster. Remember, just because you think you've improved performance doesn’t mean you actually have. It’s important to always benchmark your changes after you’re done.
00:17:14.480
Let’s revisit the benchmark from earlier that showed that including 100 modules that read a class variable was 8.5 times slower than including one module. When running the same benchmark on our branch that added the cache, we saw that we completely eliminated the progressive growth that occurs as more classes are added to the inheritance chain.
00:17:38.560
Now including one module with a class variable has the same performance characteristics as 100 modules with a class variable. While this benchmark showed a massive improvement in performance, it’s still considered a microbenchmark, which means that it doesn’t represent real-world scenarios.
00:17:53.280
We needed a benchmark that would be more representative of a real application. So we ended up measuring the performance using RailsBench, which is a Rails application that can be used to measure requests per second in a more production-like environment. It’s definitely not as good as running it on GitHub production hosts, but it can tell us a lot more than microbenchmarks can.
00:18:32.239
With RailsBench, we saw that Rails 6.1 without a class variable cache could perform 615 requests per second. We then did the same measurement using our cvar cache branch and found we could perform 657 requests per second with the addition of the cache. That means we’re getting 42 more requests per second, which works out to about a six-to-seven percent improvement in request times.
00:19:12.560
These benchmarks demonstrated that the cvar cache is faster, not just in my microbenchmark scenarios, but it also has a real impact on application performance. Through this work, Aaron and I learned a ton. At first, adding a cache to class variables seemed like a relatively simple task, but it took about six months to actually get something functioning that we were confident was not full of bugs.
00:19:56.880
I remember at one point, we thought we were ready to open the pull request for the cache. Aaron had a dream that we had a bug in the overtaken behavior, and it turned out he was right. The complexity of class variable inheritance was why making this change was so difficult.
00:20:39.520
So we finally got the bug fixed, tests passing, diagrams made, and we were confident our changes were good. We opened the pull request and feature request issue, then waited for commentary. After some discussion on the issue, I started to get really nervous because it became clear that the Ruby team wasn’t sure they wanted to merge the feature.
00:21:09.760
Of course, everybody appreciated our hard work, but as I said, class variables are very complex, and by adding a cache, we increased that complexity. Eventually, we did get it merged, but I want to take some time in this talk to dive into what concerns the core team had and how we navigated negotiating getting this change merged anyway.
00:21:42.000
Fundamentally, every change in open source has trade-offs. Let’s be real: every change to code everywhere has trade-offs. I'm sure you’ve experienced this at work or in your own jobs trying to decide whether it’s worth the risk of fixing that technical debt or just ignoring it and building your future.
00:22:31.520
Maybe you opt to add complexity because you think you will get back to fixing it later, and you never do. Or maybe you decide to not remove that outdated gem because you don’t have time, or perhaps you write a monkey patch in your application instead of sending an upstream patch because you didn’t want to wait for a release. Every day, we weigh trade-offs in our code regarding complexity against one another when making changes to our codebase.
00:23:11.440
Open source maintainers have to do the same thing; they need to weigh the trade-offs of whether a change is worth it. To Aaron and me, adding a cache seemed like a clear win. Class variables are slow, so we made them faster, making adding a cache a net win, right? What trade-offs could there possibly be?
00:23:52.560
However, any change in Ruby, especially one that involves a feature as complex as class variables, means even small changes have trade-offs that maintainers need to consider. The first trade-off the core team was concerned about was that our change increased the complexity of class variables.
00:24:08.240
As you saw today, class variables are already super complex. So, by adding something even as simple as a cache, we made them even more complex. Now, in addition to the existing complexities of inheritance and overtaking, we’ve introduced the complexity of creating, updating, and invalidating an inline cache.
00:24:47.680
When creating the cache, we had to add a second hash table to track every time we write a class variable and ensure that the cache behaves properly. Class variable behavior around inheritance means that there are a lot of places that we could have made mistakes. We had to be absolutely sure that we covered every case for when a class variable could be set in Ruby code.
00:25:14.240
Another concern the core team had was that we were going to encourage more people to use class variables by making them faster. As I mentioned earlier, class variables are not a well-liked feature of Ruby, not even among the language maintainers. Some people consider them so confusing to use correctly that they might as well be deprecated.
00:25:56.640
The concern was that if we made class variables faster, we’d accidentally encourage applications and libraries to use them more. Since many Ruby programmers are not aware of the intricacies of class variables, encouraging more usage might lead to increased bug reports on the issue tracker.
00:26:27.520
Additionally, if we keep making them more useful, then they become harder to deprecate in the future. Lastly, by adding a cache to class variables, we increase the maintenance burden for the core team. What if a new case for inheritance is added, and we miss clearing the cache?
00:26:51.520
What if there’s a bug in the existing code and we’re now hiding it? And what if we’re caching the wrong information in some specialized edge cases? What are the risks to the existing applications? For perspective, consider your application at work or a library that you maintain.
00:27:14.400
Is there a feature or part of the codebase that you know better than anyone else? How would you feel if someone came along and changed all that code? If you knew it was probably a net gain, that may be the case, but now you don't understand all the moving parts.
00:27:57.680
I've experienced this myself as a Rails maintainer. I want contributions, but I don't want to introduce changes that I don't understand because, once it’s merged, that responsibility becomes mine. I completely understood where the core team was coming from, even though I felt that adding a cache was a net win.
00:28:37.760
Once we understood the concerns of the core team, we had to convince them that the change was worth it. Negotiating whether a change should be merged is tough, but it’s an important skill for contributing to open source, or any work you're doing. First, we pointed out that class variables are not deprecated and are unlikely to be removed from the language.
00:29:25.680
Even if they're an unpopular feature, while they’re complex with strange semantics, there are some cases where they are the only feature we can use. A good example of this is the Rails database configuration. We can’t use an instance variable for that; it should be global, meaning there should only be one value, with no other things allowed to change that value.
00:30:00.800
Using an instance variable in that case would break the contract that ActiveRecord relies on. Additionally, the benchmarks we provided showed a real-world impact on applications. It wouldn’t have been convincing if all we had were microbenchmarks that showed an improvement to class variables in isolation.
00:30:43.520
By getting a benchmark that showed improvements in request times for Rails applications, we demonstrated that the cache was effective in making request times six to seven percent faster. The benchmarks are compelling because they demonstrate real-world improvements, making the complexity worth taking on.
00:31:23.680
Lastly, one of the benefits of making this type of change to Ruby is that all applications and libraries get a performance boost without changing their code. All you have to do is upgrade. This is compelling because if Ruby is slower, fewer applications will upgrade, but if Ruby is faster, more applications will upgrade.
00:32:12.000
Also, it’s more enticing to upgrade if you don’t need to do any work to get free performance improvements. With these three points, we convinced Mat and the Ruby core team that it was worth trying the class variable cache in Ruby, and it was merged in June, being released in the preview earlier this week.
00:32:49.280
Negotiation isn’t about being agreeable all the time or acquiescing to what a maintainer wants; it’s about understanding the concerns they have and working to demonstrate the value of your change around those concerns.
00:33:00.800
When negotiating with others, take time to understand the concerns that maintainers have because they end up maintaining that code. Those concerns are valid. Take time to consider how you can make the change less risky or complex, and think about how you can prove the change is a net gain even if there’s more complexity.
00:33:41.760
Be sure that you’re willing to do the work to advocate for your change and provide evidence that any added maintenance or complexity is worth the cost. I hope that learning about our journey to add a cache to class variables in Ruby has inspired you to contribute to Ruby in the future.
00:34:13.440
I want us to contribute more to Ruby because this is how we maintain a healthy ecosystem for years to come. If you’re worried about Ruby becoming less popular or dying, the best way to prevent that is for all of us here to start contributing to Ruby.
00:34:45.440
Let’s work on making Ruby better so that applications can not only run on Ruby but can thrive and scale on Ruby forever. I know firsthand that contributing to open source can be a little scary, so I want to take some time to talk about how you can start working on Ruby as well.
00:35:11.040
The first thing you should probably do is learn C. I’m not a C expert, so while writing this talk, I actually had to go back and reread the cache code 800 times to figure out how it worked. I wouldn’t even call myself a beginner C programmer; I’m more like a C tourist visiting it.
00:35:41.440
When I want to work on making Ruby better, you want to be a C expert, but you just need to learn a little C because guess what? Ruby is written in C, and now a little assembly, so you can learn both. The more C you know, the easier it will be to figure out how to change internals.
00:36:10.440
In addition to learning C, you're going to want to understand how Ruby is designed. For that, I recommend reading 'Ruby Under a Microscope' by Pat Shaughnessy. The book dives into some of the design of Ruby's parser, BMs, and other areas that make the language unique.
00:36:44.480
The book was written in 2013, so some of Ruby's internals have changed since then, but it’s still an invaluable resource for understanding how Ruby was designed. The fastest way to start contributing to Ruby is to start contributing.
00:37:24.040
That sounds simplistic, but it's true! You can’t contribute to Ruby by merely thinking 'I’d love to contribute to Ruby;' you have to do it. Making changes to a programming language that’s used by hundreds of thousands of web applications can be overwhelming, but contributing to Ruby doesn’t mean you have to implement a new JIT to make Ruby better.
00:37:43.440
You can make small changes like documentation updates, fixing bugs you find on the issues tracker, or by benchmarking and improving performance. I've heard people say things like 'Ruby is slow because it’s Ruby.' But really, if there's something slow about Ruby, it means no one tried to make it faster yet.
00:38:00.560
So the person who could make it faster might be you. I hope this talk has inspired you to contribute to Ruby and look for ways to make the language better. When we contribute to Ruby, we make the language better, cleaner, and add features for everyone.
00:38:28.800
Contributing to Ruby isn’t charity and shouldn’t be treated that way. It’s not a donation to the community, and I don't do it just for fun, although it is fun! But I contribute to Ruby because I want it to thrive as a language. I contribute so that GitHub can run on Ruby for five, ten, or even one hundred years.
00:39:16.560
I contribute so Shopify can run on Ruby for 100 years. I do it so your application can run on Ruby for 100 years, and yours and yours and yours. I want you to contribute to Ruby for the same reason — to ensure that it can support, grow, and scale your application for 100 years.
00:40:00.320
Ruby can’t disappear if we don’t let it, and it can’t go anywhere if we’re actively working on making it better. So let’s go make Ruby better together!
00:40:37.840
I don't think I have time for questions, so feel free to ask them on Discord, and I will answer later as the day goes on. All right, thank you!
Explore all talks recorded at RubyConf 2021
+92