Talks
Refinements - The Worst Feature You Ever Loved
Summarized using AI
Refinements - The Worst Feature You Ever Loved
by Paolo "Nusco" PerrottaIn the presentation "Refinements - The Worst Feature You Ever Loved," Paolo Perrotta delves into the complexities and controversies surrounding the Ruby programming language's feature called refinements. Introduced as a solution to issues with monkey patching, refinements aim to localize changes to classes without affecting the global scope. However, their adoption has been fraught with challenges, leading many in the Ruby community to largely ignore them despite their initial promise.
Key Points Discussed:
- Origin of Refinements: Refinements originated from a proposal made by Ruby's creator, Matz, five years ago to address concerns about monkey patching, which allows developers to modify existing classes globally, often leading to unexpected bugs.
- Controversy in the Community: When refinements were up for discussion again before the release of Ruby 2, opinions were split. Critics voiced that the feature could severely harm the language, while supporters advocated for its integration.
- The Nature of Monkey Patching: Perrotta outlines monkey patching as a common practice in Ruby, allowing developers to redefine methods. While it has its benefits, it can create significant problems due to its global scope, impacting the language's stability and predictability.
- Technical Explanation of Refinements: The speaker explains that refinements work by defining a method within a module and using it in a local context. This allows changes to be limited to specific parts of the code.
- Complexity and Confusion: Despite the aim for cleaner modifications, refinements can lead to confusing situations, especially when a method's behavior changes based on context. This unpredictability challenges the expectations of developers, leading to potential errors in larger projects.
- Performance Issues: A major downside discussed is the performance impact that refinements can have on Ruby, particularly with method caching, as they prevent the typical optimizations in method lookups, potentially slowing down the language overall.
- Lexically Scoped Refinements: Due to the found issues, the final implementation of refinements in Ruby is lexically scoped, meaning that refinements only apply within certain declared scopes, which some believe diminishes their original purpose.
Overall, Perrotta concludes that while refinements offer a structured approach to modifying Ruby's behavior, they come with several caveats that may discourage developers from using them in favor of the familiarity and flexibility of monkey patching. His talk emphasizes the importance of understanding these trade-offs to make informed decisions about coding practices in Ruby.
00:00:10.900
Hello, so this is from the room decor mailing list or a bug track. A few years ago, about five years ago, Sugar Maya wrote an issue about adding a new feature to the language.
00:00:24.369
Matz had kind of talked about that for a short while. It was called refinements. Five years ago, he made the proposal, wrote an implementation, and then nothing happened for like two or three years.
00:00:39.230
About three years ago, right before Ruby 2 came out, the discussion started again, and it became a huge discussion. This is not my normal voice; hello again. Can you take these off? I could do this all day.
00:01:06.500
So, right before Ruby 2 comes out, the discussion about refinements starts again, and it becomes contentious. On one side, there are Matz and the submitters, and on the other side, there are people from the community who have a presence in the community, like the J-O-B people.
00:01:14.300
Some of these people are saying, 'Hey, this feature is dangerous; it might kill the language.' That's what they said, and they fought about it for a while. Ultimately, refinements got toned down somehow and made their way into Ruby 2.
00:01:52.010
So, that was five years ago when it started, about three years ago when this was closed, and now we have refinements, and everybody's using them, right? I mean, if you use the refinements, raise your hand.
00:02:05.570
This was a total failure. It was supposed to be one of the most important new features in the language since the very beginning. It kind of faded away, and nobody knows why. If you ask people, they will say, 'Oh yeah, nobody's using them,' but nobody really knows why. It's hard to verbalize.
00:02:40.730
So, this is what I want to do now: I will take half an hour to tell you all I know about refinements. Not all of it literally, but a lot of stuff, so you can make up your mind. At least then, you can keep avoiding them, but you will know exactly why you're avoiding that.
00:03:00.299
Okay, it's going to be a technical talk, so let's start from the very beginning. Why do refinements exist? What's the problem that refinements were set to solve?
00:03:11.400
In Ruby, you can do this: you can open an existing class, in this case, the String class, and put a new method in there or redefine a method that already exists. In this case, we have a new method called display that prints out the String itself or tosses or destroys itself. It's all happy and full of nice little Japanese cycles.
00:03:31.049
Now, when you say, for example, 'Hello.display,' everybody's super happy, right? So, this is a feature that some people call dynamic class scope, or open classes; most of us just call it by a term that is vaguely negative: monkey patching.
00:03:55.470
Okay, that's what monkey patching is. So, what is monkey patching? Why do we have monkey patching? Well, there are five or six use cases you can probably think of where they change the language.
00:04:28.530
Ruby is defined by this. It would be a very different language without monkey patching. It would feel like Python. Python is a great language, but it's not Ruby anymore.
00:04:41.160
Amongst all the use cases, you can identify a couple of main use cases that you see all the time. First, convenience methods in Rails, for example. You can say 'one hour plus twenty minutes,' like this. Of course, hour and minutes are methods.
00:05:06.480
They were added to numbers, whether they are fixed numbers or any kind of number class. It's not necessary; you could say 'I hour' with parentheses, for example. You could have regular functions or top-level methods.
00:05:34.530
But it looks good, and it's important to write code that looks good. Do we care about that? It's part of the language, so you cannot do this without monkey patching.
00:06:05.520
Here’s another use case that goes a few steps further. Once you start going crazy with monkey patching, you build structures that actually look like a specific language.
00:06:14.370
This is an example from RSpec. I have it here because it's the textbook example of a domain-specific language. Even though this is not valid RSpec anymore, this is an old version. Still, if you look at it, it looks great. It reads like English; it's very expressive.
00:06:43.190
You cannot do this without monkey patching because if you look at it, 'should' must be a method on some object, I don't know, integers, a basic object, the kernel module, or something like that. It’s a method somewhere that was defined with monkey patching.
00:07:20.610
We care about monkey patching; it's one of the basic features of Ruby. It's awesome. So, what’s wrong with monkey patching? What's the problem? The problem arises, for example, when I define a method called 'display,' thinking I have a nice new method.
00:07:53.420
But there is already a method called 'display' in Ruby. I bet you never used it, but it's there. So I just broke Ruby by overriding an existing method that I didn't know about.
00:08:19.500
That's why monkey patching is often considered problematic: it’s global, and global is bad. We don't want global changes in programming, especially in large programs.
00:08:42.640
If you make a global change, you never know when it’s going to cause a problem. Maybe somebody else defines the 'display' method in a gem that I’m using.
00:09:02.100
How can I possibly know? I only find out when it breaks. So, what's the solution to this problem? That's where refinements come in useful.
00:09:37.490
The idea of refinements is similar to monkey patching, only local. It's not global anymore, so I have control over it. Here's how refinements work in less than two minutes.
00:10:10.290
First, it's a two-step process. First, you define the refinement, and then you use it. To define it, you need a module. We use modules for various things such as mix-ins, namespaces, and now we also use modules to define a refinement.
00:10:45.149
Inside the module, I have to use the new keyword to redefine methods. I can't remember whether it's a keyword or a method, but it’s a thing. And I say redefine this class.
00:11:11.919
Now, inside the class, I do exactly the same thing I did with monkey patching; I define new methods or redefine existing methods. The difference is I'm not polluting the global namespace. It's not even active yet.
00:11:39.390
This is just a declaration. Then, I need to use the refinement. To do that, there's another keyword, but I can't remember which it is. You can use the 'using' keyword in multiple contexts.
00:12:00.690
For example, you can use it inside a module or a class. The class is just the module. I can say 'using StringExtensions,' and from the moment I use 'using,' the refinement becomes active.
00:12:19.980
So, I can say 'hello' twice, and this only works after the 'using.' It works from right after the 'using' to the end of the class or module.
00:12:45.790
I can also use it outside of a module or class. In this case, the refinement stays active from right after the 'using' to the end of the file.
00:13:06.850
For completeness, I can take a string that contains Ruby code and pass it to 'eval.' I can have a refinement in there as well; it’s going to stay active from right after the 'using' to the end of the string.
00:13:32.130
So, it looks like there is no reason not to use this wonderful feature, right? But even if this looks super simple, this is only part of the story. There is something that's not immediately visible going on here.
00:14:06.080
What is the problem with this solution to the problem of monkey patching? We said that in Ruby, you can use a refinement inside the class, and it stays active until the end of the class. But what does 'end of the class' really mean?
00:14:44.770
What if I reopen the class? Is the refinement going to stay active there? This is not the only case; there are multiple ways to reopen a class.
00:15:00.110
Instead of literally reopening the class with the 'class' keyword, I can inherit from the class. It’s kind of like the same scope. Will the refinement work there or not? For people who like to play on the edge of consequences, there is another cool way to get into a class scope in Ruby, using the class cVAR keyword.
00:15:58.210
It's a block, and the block gets executed inside the scope of the class. Again, it’s all kind of like the same thing. Once again, is this going to work? I expect it to work because that’s how Ruby works.
00:16:16.430
I have to assume it does because, in general, Ruby is a very dynamic language. You close the scope, you reopen the scope, you find all the stuff that was in there, including instance variables and methods.
00:16:55.350
Now, I expect refinements to work in these cases. This is what we call dynamic scope. It's one of the defining features of Ruby. It doesn’t seem like it's going to pose any danger; it looks pretty harmless, but there are a few issues.
00:17:27.229
First, it makes for some confusing code. How confusing? I will let you judge. Here’s a pretty concrete example. I have a method called 'add,' and it calls another method.
00:17:52.580
Then, I execute this method in multiple contexts, in the context of one class and in the context of another class. Finally, I use a refinement module that the comment says refines the plus method. Then I call 'add' again. In the first case, I get 2, as I expect.
00:18:46.900
No one has redefined the 'add' method or the 'plus' method. In the second case, though, some other class does something with the refinements that I wasn’t expecting.
00:19:09.190
When I say 'plus,' it converts the numbers to strings and then concatenates them. This was probably written by a JavaScript developer. So now I have something I wasn’t expecting there.
00:19:43.079
This other refined version of the class seems to prefer floating-point numbers, so it isn’t going to return an integer; it’s going to return a floating-point number. Okay, this looks fine in a small example.
00:20:01.920
But think about this on the scale of a very large project. Every time you use a method in every context—even contexts that are very close to one another—you might get a completely different result.
00:20:22.130
The only way to know what that stuff is doing is to check all the refinements that get used inside the scope. So, is this evil or not? People have different opinions about it.
00:20:49.159
Some people say this is evil; it’s implementation leaking into the interface. You never know what you’re doing, and this is super confusing. Other people say, well, this is just Ruby. You can do something similar with monkey patches already.
00:21:22.890
If you try really hard, you can confuse me. I want to know what's going to happen unless I look inside the code. So, there is some debate. One big concern people often have is that not only is this confusing, but it can also be dangerous.
00:21:55.660
If I don't know what’s happening to my code, I can easily execute malicious code. The second camp counters, 'Sure, who cares? You can do that with monkey patching as well.' So, this is one potential issue.
00:22:32.520
Another potential issue is that refinements have corner cases you might not expect. I have more than one, but I don't want to make this too long. Here’s one: I open Pry and try to define a refinement method.
00:23:08.440
Then, I call 'hello,' and it doesn’t work. It works in a file, but it doesn’t work in Pry. Can anybody tell me why this happens? I couldn't think of it on my own. Ok.
00:23:43.120
That proves that you shouldn't update your slides the night before the presentation. But imagine that I did put the 'using' in there; it still doesn't work. Why doesn't it work?
00:24:05.700
If I do it in Pry, while it works in a file? Well, Pry essentially takes each line, treating it as a string of code, and executing it. You might not expect this, but this is exactly what happens, which is infuriating when you copy-paste code.
00:24:53.530
You assume it should work, and it doesn’t. Finally, the real issue with refinements that make some people consider them evil is performance.
00:25:14.630
When you call a method, what Ruby does is it looks into the class of the object you’re calling the method on and starts looking for the method. If it doesn’t find it, it goes into the superclass of that class, walking up the chain of ancestors.
00:25:51.120
These chains can get really long. For example, when you call a method on an ActiveRecord object, there are often dozens of ancestors; it’s a very long chain.
00:26:19.470
This happens every time you call a method, which is basically what you do all the time in Ruby. So, it actually consumes a lot of the Ruby interpreter's processing power.
00:27:01.450
The process of walking up the chain can be slow. When you call a method again on another object, you don’t want to walk the chain again, right? So, there’s an optimization in the Ruby interpreter called method caching.
00:27:45.860
The first time Ruby finds the method in this chain, it caches that position and skips right to the method. It only validates the cache when the chain of ancestors changes.
00:28:25.160
This is crucial. It is a big deal in Ruby; it's the reason Ruby is fast. Except, once you introduce refinements, you can’t do this anymore.
00:29:06.649
Why? Because when I call a method, I don’t know where the method is—it depends on the context from which I’m calling it. If I call it in another context, it might have different refinements.
00:29:55.809
Essentially, the whole concept of method caching becomes irrelevant; you have to look for the method every single time. This means that, for example, some people estimated that we’re going to see a twenty percent drop in execution speed for Ruby, not just executing refined code.
00:30:24.919
So, just having dynamically scoped refinements makes Ruby slow down by twenty percent even if you’re not using refinements. So, when we recap the trade-offs, dynamically scoped refinements are super cool.
00:31:05.780
They fix monkey patching and are local. They are great except they make for some confusing code and have hard-to-pass corner cases that might surprise you, along with a significant performance impact.
00:31:42.780
Is it worth it? This was the debate right before Ruby 2 was released, and in the end, the camp that won said no, it’s not worth it. This is not what we want.
00:32:25.950
What did they do? They decided to change refinements so that they were not dynamically scoped anymore. Now they are lexically scoped. This change convinced people that, okay, this feature can be kept.
00:32:48.990
We had a problem; we found a solution. We found the problem with the solution, then found the solution to the problem with the solution.
00:33:19.950
You remember why we decided to introduce refinements in the first place? I told you at the very beginning: we had use cases where we wanted monkey patches, but we wanted them to be local rather than global.
00:33:56.410
For example: convenience methods. Imagine you’re in an ActiveRecord or Rails class of some sort, and this class is applying refinements. You want to have new convenience methods, but with the current lexically scoped refinements, you cannot do this.
00:34:36.640
There’s no 'using' there. You need to put that 'using' in there, but it’s cumbersome because you cannot write dozens of 'using' in your code. It takes away the convenience.
00:35:09.490
And that's why Rails doesn't do that; they keep monkey patching everything. Domain-specific languages also work differently now, because you need a 'using' to make it work.
00:35:41.710
This is a really weird state of affairs. We came up with a solution to a problem, and after much back-and-forth, we essentially made the solution so tame that it doesn't address the problem anymore.
00:36:17.360
I’m not saying refinements are not a good feature, but all this talk and complex reasoning is probably the reason why people say, 'I don’t care. I don’t even want to understand this stuff; I will just keep using monkey patches.'
00:36:41.790
After all, that's what we've been doing for the last few years. All right, nobody died using monkey patching. So, the question is: was it worth it? Here are the trade-offs.
00:37:13.919
With lexically scoped refinements—the ones we have now—they kind of fix monkey patching at no huge cost: no performance issue, no huge confusion.
00:37:22.390
Yes, there are still some downsides. You’ll have to call 'using' every time you want to use it, which might be annoying to some.
00:37:36.500
It’s up to you whether you think it’s worth it. But at least now you know what it’s about, and you can take an informed decision.
00:37:47.210
Okay, and that’s what I wanted to talk about. Thank you! By the way, this book is great.
00:38:08.240
That's wrong. I'm honored. Any questions?
00:38:13.370
I'm pretty sure it doesn't; it's still lexically scoped. Even if it did, it would be so magical that I would shy away from it.
00:38:34.190
You know, that’s when you really got to forget about this sort of magic happening when reopening a scope. I don't know, but in any case, it’s not going to work lexically.
00:38:54.780
It’s about text: either you see a 'use' right in there, or it’s not going to work. Yep, I don't write much Ruby.
00:39:12.740
I’m not the best Ruby developer to be honest, so it might be that I just don’t find convincing use cases.
00:39:25.500
I don’t use meta programming that much either. Still, you should redo Nick.
00:39:48.820
Oh, the irony! That basically erases the major use case for pretty much every new feature.
00:39:54.900
Yeah, you're right. All right, are you using it in Trailblazer? No, I mean refinements.
Explore all talks recorded at rubyday 2016
+1