Talks
Module Magic
#ruby-metaprogramming
#ruby-internals
#ruby-programming
#ruby-kaigi
#event-driven-programming
#code-organization
Summarized using AI
Module Magic
by James Edward Gray IIIn the presentation titled "Module Magic" at the LoneStarRuby Conf 2009, James Edward Gray II dives deep into Ruby's modules and their utility in programming. The session aims to enlighten attendees about the mechanics of Ruby modules while sharing personal experiences from his trip to Ruby Kaigi in Japan.
Key points discussed:
- Introduction: Gray opens with a personal note about the lack of logarithmic graphs in his presentation and mentions his previous talks at LoneStar Ruby Conf, establishing a connection with the audience.
- Ruby Kaigi Experience: He encourages attending the Ruby Kaigi conference in Japan, highlighting benefits such as cultural exposure, networking with Ruby enthusiasts globally, and understanding the Japanese conference format, which includes real-time translation.
- Exploration of Ruby’s Module System: The core of the talk centers around Ruby modules, starting with the basics of creating mixin modules and extending classes with methods. Gray uses code examples to illustrate how method resolution works with modules in Ruby.
- Example Interpretations: He presents several pieces of Ruby code, emphasizing how method calls function in Ruby when using modules, specifically highlighting the print order of methods and the importance of understanding the method lookup path.
- Creative Use Cases for Modules: Gray discusses unique cases for applying modules, such as auto-generating error classes, using modules for namespaces, and modifying object behavior in creative ways.
- Incorporation in Ruby Projects: The presentation culminates with practical advice on utilizing modules to enhance flexibility and maintainability in Ruby code, suggesting a shift from traditional class inheritance to more modular designs.
- Conclusions: The discussion wraps up with Gray encouraging attendees to experiment with Ruby’s method lookup and implementation of modules, emphasizing the importance of creative and thoughtful usage for effective coding.
Overall, Gray's presentation intertwines technical knowledge with personal anecdotes, providing valuable insights into both Ruby as a programming language and the culture surrounding it in Japan, making it an informative session for Ruby developers.
00:00:20.320
Okay, and now we're going to hear from James Edward Gray II.
00:00:34.320
Okay, so I am going to talk about a few things. Before I do, for those of you who don't know, here are some of the things I've done. I won't read them to you and insult your intelligence, but I do have a series about M7N and Ruby 1-N on my blog. So for all of you who weren't at my training yesterday, you are obligated to go read my series on M7N. You have to check that out. That's me on Twitter.
00:00:58.559
Today I'm going to talk about something different. Like Glenn, I've spoken at every Lone Star Ruby Conf so far. I did 'Heroes' the first year and 'Battlestar Galactica' the second year. But it hurt my feelings when I did 'Battlestar Galactica' and over half the room had no idea what I was talking about. It's like, what kind of geeks are you people? This year, I decided I'm not going to do a TV show. I'm going to tell you about Ruby Kaigi, which is where I went this summer in Japan.
00:01:34.000
So, I'm going to try to encourage you to go see Ruby Kaigi. There are lots of great reasons to go, such as seeing how the Japanese conduct conferences. That's a big plus. Also, if you give a speech there, they translate it as people ask you questions, so you get extra time to think about what you're going to say. You don't sound nearly as stupid. It's a great opportunity to meet Rubyists from all over the place. I met some from Japan and others from various places. And, of course, you get to see Japan! That's the reason to go to Ruby Kaigi.
00:02:07.919
These are some of the people I met: that's Floran from Germany, that's Kazuki from Kyoto, and Soo, who translated my slides into Japanese, so he was a big help. You get to see Japan's amazing temples, like the Nanzen-ji, which I think is one of the temples. The views in Kyoto are amazing; you can also see Sumo wrestling, like I did. That's the guy in red, Yamamoto Yama. He's over 500 pounds, and this picture was literally taken while he was picking that guy up and flinging him out of the ring.
00:02:37.920
You can't get anything like that here; it's just great. I could do 45 minutes of this, and I have a lot of pictures. I took 800 while I was there! This is a view of the streets of Tokyo, shopping in Tokyo, and this is the view out of my window in Tokyo at night. So, you can see the nightlife in Tokyo. This is a tea house in a garden that I visited.
00:03:05.000
Okay, we better talk about some Ruby, or they're really not going to let me talk next year. So here's some Ruby. This is something I suspect everybody in the room knows: this is just a simple mixin module. You define a module, put a method in it, include that module in a class, and then instances of your class have access to that method, right? No surprises there. Everybody knows how this works, right? You can call that method on that class. No surprises. Good, we talked about Ruby. Let's go back to Japan.
00:03:51.239
So this is a sand garden in Japan. You see these all over the place. It's like a meditation garden, and they design the sand in certain ways that encourages contemplation. I'm not even sure what you contemplate—maybe your relationship to the things in the sand garden and stuff like that. Some of them are kind of strange, like this cone one here. These meditation gardens can be found in all the temples in Japan.
00:04:38.000
Here's a piece of Ruby code I think is worth meditating over. It's actually a pretty interesting piece of code. Let me walk you through it. It's just a normal class that has a method called 'call.' When you call it, it prints out the class name. I apologize for the use of eval, but the slide doesn't have much room for code. It just defines three identical modules: A, B, C, and D. They each have a call method which prints their name and then calls the next module up the chain. That's all they do: print their name and call up.
00:05:40.480
Then we build a child class of that original class and dump all three of those modules in there. The child class also has a call method that just prints its name and calls the parent class. So, it's just calling up the stack. Then I make an instance of that class and call it.
00:06:49.800
So, everybody got what that does? Does everyone kind of understand that? Raise your hand if you think you know what this prints. How many people? Oh, not very many. We don't know what this prints? Come on, it's only five things! No? We don't know? Okay, see, this is why you meditate on this code! It's very important.
00:07:01.759
This is what it prints right here, and it shouldn't be too surprising. The bottom class prints first because we called that method on the bottom class. Then the three modules print, but note that they print in reverse order, right? I included them in the order A, B, C, D, but they print D, C, B, A. Then the parent class at the very top prints last. This one piece of Ruby code explains about, I don't know, probably 80% of Ruby's method call mechanism, and if you can get the hang of this, you can do great things.
00:07:38.160
Okay, so I'm going to show it a different way. This is the inheritance of that slide you just saw. All I did was ask that object for its ancestors. Under Ruby 1.9, if you run it under 1.8, it would look exactly the same, except without the basic object at the beginning. But you can see there's that class E, and it says it inherits from those modules we included: D, C, B. Then there is my parent class, and then you've got Object, and Object inherits from Kernel, which by the way is a mixed in module, just like what I showed you. Then it inherits from BasicObject.
00:08:14.000
Okay, that's boring! Let's go back to Japan. You can see all this great water in these gardens in Japan. It's the clearest stuff you can see right through, and check out the fish swimming around. You can put your hand in, and you can just see it because it's so ridiculously clear. They have this water all the gardens in Japan, so you can look beneath the surface and see what's underneath.
00:08:51.920
Let's see if we can go beneath the surface of that example I showed you. Just one more level. This is pretty much the same thing. The top two chunks are the exact same thing, but the bottom is different. I totally got rid of the child class. Instead, I made an instance of the parent class and then extended that instance with those modules. Then, I do the call.
00:09:06.919
Okay, okay. We had about, I don’t know, ten people last time who thought they knew what the last one prints. There are only four letters this time. Raise your hand if you think you know what this prints. Yeah, that’s about three people. No, four— I think I see four.
00:09:38.760
Okay, so, this is what it prints. You can see the modules come first: D, C, B, and again in reverse order of what they're included. Then you get the parent class. If you get this one now you’ve got like 100% of Ruby’s call mechanism. You can understand how Ruby handles methods. This one, if you look at the ancestors, it's even trickier; look at that. It says that when you call ancestors, it says it inherits from the modules first, and that doesn’t seem to make sense.
00:10:09.000
Wouldn’t a class have to come first? Actually, it does, there’s an invisible class right here in front of them that Ruby's source tends to refer to as the Singleton class. It's invisible, and it’s in the front, and then those modules are stacked up behind it, just like they were behind the child class. Then you have the parent class.
00:10:29.800
This actually works the exact same way. The only difference is that there’s an invisible class involved this time. We can actually make that show up if I do the exact same example, but this time I tuck a method into that invisible Singleton class. Then when I print it out, look, it shows up! There’s the invisible class, there's the modules right behind it, and then the parent module. You can call all the way up the call stack and get back to that parent class.
00:10:54.000
So, that's the way modules work. As you realize that, you can do amazing things because now you realize you can put methods behind your class and also put methods in front of your class, right? If you want to change behavior or modify behavior, this also told us what extend is. Extend is actually just a shortcut; it means basically this—open the invisible class and include that module. It's a shortcut.
00:11:37.279
Another thing we use modules for is different things. This is using modules as namespaces. This is Logger in Ruby's standard library. I imagine it's a class most of us have seen at least a little bit. It has this Severity module inside that has all these different debug levels. It tucks these constants away inside the module, and interestingly, it includes that module inside the Logger to get access to those constants. You can move those constants around with that typical include mechanism. If you were building some programmatic JSON representation, you might need to make classes like Array and Object, which obviously you can't do because Ruby has an Array class and an Object class. So you could tuck these away inside a module and they would be in a different namespace.
00:12:15.320
Okay, let's go back to Japan because this is a lot of code stuff and it's very boring. This is food in Japan, which is awesome! That’s like one of the reasons to go. Although, don’t ask Dana because she doesn’t eat fish and that’s not a good thing to be in Japan. This was a block of ice with food sitting on top of it. If you lift that metal thing off, there’s a bowl hollowed out in the block of ice. You can kind of see it in there, and there's a noodle broth soup sitting inside the ice bowl, which is pretty cool. It’s all kinds of crazy stuff. I wouldn’t know what half of the things I was eating were, but it was great; I ate them anyway.
00:13:01.800
Desserts in Japan look great, don’t they? Interestingly, if you saw this dessert here, you would probably think it’s really, really sweet. You take a bite, and it would be really, really sweet. But if you see a dessert like this in Japan, odds are it’s not very sweet. They don't like things as sweet as we do. They tend to be more bland than what we eat here. This is Sashimi from the Tsukiji Fish Market, the world's largest open-air fish market. So, fresh fish and fish eggs and stuff—that's great! Japanese food is all about presentation, right? They have this great presentation when they put it in front of you, and you’re just like, wow, that’s awesome.
00:13:52.960
Sometimes we have code that’s about presentation, too. There’s a Math module that comes with Ruby. You can use it in two different ways. If I’m writing a distance function, I can just call into it like this: here's my distance function. Or I can choose to write it like this: I can extend my class with the Math module and then refer to that method without calling it into Math, which means that the module has to be both a class method on Math and an instance method on Math, right? It has this kind of dual interface.
00:14:43.200
The Math module actually does it with Ruby's module function, which is handy. I believe you can do the same thing even easier because it follows the same rules we understand for modules. You can just put your own module inside your ancestor tree. Just extend self and you’ll put your own module behind your ancestor tree. In this example here, I’ve just called the little log method I wrote directly on the mini logger object. Or over here, I’ve included the mini logger module and then I can override logger and call log as an instance method.
00:15:10.400
So it gives me that dual interface, the same thing you see like in the Math class. One of the awesome parts about running around Japan is they have these shrines everywhere because Shintoism is so big. This shrine literally has a gigantic store on the right of it and a gigantic store on the left of it, so you probably wouldn’t imagine that from the picture. But they’re hidden away all over Japan, and part of the fun is running around Japan and looking for these little shrines. This one is on the side of an access road, so you got trucks coming around behind, and there's a little Shinto shrine sitting on the side of an access road.
00:15:58.960
This one is actually at a temple, but it’s this stairway up to the side, and you kind of take this around, then you just dead-end at this shrine. They’re tucked away everywhere; it’s like these little bits of magic all over the place. You can use modules for the same thing—to tuck away little bits of magic and keep them from running into the other parts of your program. This is a trick I use sometimes. Some people don’t like it by the way, but I think it’s kind of fun. You can auto-generate error classes as you need them, so you just build some base error class, define const_missing, and whenever you try to access some constant, like SaveError, it inherits a class from that and sets it to that constant for you.
00:16:45.680
So, you can generate error classes as you need them. This is a bit of magic because I had to override something, and if I did that at the top level, then that would be a really dangerous hack because you could just invent classes all the time. But because I tucked it away inside a module, it’s not really a big deal. The whole point of this module is to contain that magic. Now, I'm going to give you some examples of just different ways to use modules. These will maybe get you thinking along lines you may not have traditionally thought about.
00:17:44.000
This is just some unusual ways to use modules. I actually had this code in a project recently. The point was I wanted a configuration object that was pretty much just an OpenStruct, but I also wanted it to have a couple of behaviors, like being able to load itself from a configuration file or something like that. I wanted to be able to put a few methods in it, so I did this clever thing, which Ruby totally let me do. Then eventually, this was part of our public API for this project, and my boss said, 'You’ve got to document all this!' I said, 'You can’t document this. Rdoc doesn’t read it, but Ruby will read it. However, the Rdoc parser sucks, and it won’t read it right.'
00:18:36.100
The cool thing is the Rdoc parser sucking might be a feature! If our doc won't read it, it’s probably a bit too clever. This is just how it's used with the method calls and setting values, just like at OpenStruct. Okay, we can rewrite the same thing using a module. All we need to do is build some module with the extra methods I want to include, and notice I can document this just fine now. Rdoc doesn’t care; it can read modules. When I create the OpenStruct object, I just extend it with that module, boom, all the behavior gets put into that one instance and now there’s still just that one instance that has that modified behavior.
00:19:10.640
So, it’s the same thing only Rdoc and I can both read it now. I think we should be doing more of this, which is kind of what I think Dave Thomas was talking about when he said he was doing less building with classes and more building with modules. You can make these individual objects, and then you can just mix some behavior right into them.
00:19:53.800
One of the cool things about Japan is everything’s just so classy. Like this, I don't know, is that a cat? It's a cat statue. I’m pretty sure it’s a cat, but what's awesome is they have this legend about this statue. There’s another one right next to it, and they are at a shrine in Ueno Park. There’s a big pond in that park, and they believe that these statues are so lifelike that at midnight, they sneak off and get drinks from the pond!
00:20:18.960
They have all these cool legends. This is the grave of the 47 Ronin. Does anybody know the story of the 47 Ronin? Raise your hand if you know that story. Okay, yeah, a few people know. One of their masters drew his sword on imperial ground, so he had to be put to death. His 47 samurai became 47 Ronin, and they snuck off and hid in the population, planned, and plotted their revenge until everything was perfect. Then they stormed the enemy lord's castle that had provoked him and killed him.
00:20:49.720
Then the emperor was impressed by their devotion, so he allowed them to all commit ritual suicide instead of being put to death because ritual suicide is honorable, and being put to death is not. So, they have this grave, and people come and put incense on their graves, as well as the individual resting places of the different members.
00:21:31.000
This is a Torii gate. You’ll see these all over Japan—spirit gates where you go through and it’s believed you’re purified as you go through them and enter these holy places. Everything's real classy in modules. We like to have our class methods too, but Ruby doesn't give you that option by default. But that's okay! We figured out a way around it!
00:22:13.680
This is a popular pattern. It was around before Rails, but Rails probably made it popular. When you want to mix in a module, you can basically break it down into two submodules: class methods and instance methods. Then you can extend your class with the class methods and include the instance methods in your class, so you get the class-level stuff and the instance-level stuff.
00:22:50.800
So, basically, you get a dual-level mixin. That's cool! Another thing you see in Japan is strange stuff. Like, I’m not sure if this ad makes me want to go eat bananas. It may be the goal! I’m not sure. They also build gigantic anime statues that tower over their city, which was totally awesome, by the way! I have a picture of me at the foot of one; come see me later.
00:23:47.139
So, yeah, big giant anime statues! At Ruby Kaigi, you can see Aaron Patterson and his 'laugh ninja' that literally stood behind him as he gave his talk and held up signs so the Japanese audience would know when to laugh at his American jokes. Very clever! I'll have a laugh ninja at Lone Star next year!
00:24:38.800
You can see strange things, but you can also do strange things with modules. Here are some strange things I may not have thought of: DRB Undumped. Does anybody know this module? You can include it in a class you’re going to send over DRB, and it tells DRB to send a proxy instead of actually sending the object. So your object really stays on your side and doesn’t go. This module, you can actually see a method here, but this method is never used. It's just added as a kind of protection to ensure it’s never called. It raises an error.
00:25:09.280
This is actually an empty module—it has no purpose. The purpose is to tag something, to name it or label it. So you include this module in your object, and now your module is a new kind of thing; it’s also a DRB Undumped. That has meaning to DRB; it can identify it. So even empty modules can be useful. Also, you can do some object editing. You can't generally do this in normal Ruby because some of these string methods are designed to return nil if they didn’t make any changes, the bang versions of the methods.
00:26:26.080
So, you can’t chain together the bang methods because you get a nil and then you try to call another method on it, and it blows up. But we could actually use a module—not to include methods—but to edit the object! So this is a module, and I tie into the extended hook. This is a hook that Ruby calls whenever you extend an object with your module, and it’s going to pass in the Singleton class of the object you are extending. You can take this as an opportunity for free editing.
00:26:55.230
Whenever a string is extended with this module, I run through the method list quickly and redefine all the bang methods so that they return self. Because I define them in the Singleton class, we learned before with the method lookup that those come before Ruby's traditional methods. So I override all of Ruby's traditional methods, and now you can safely chain bang methods together. Is this a good idea? Probably not, but it’s interesting that you can do it.
00:27:52.720
Here’s kind of a summary of just some things I've noticed about my usage of modules and some points I think we should consider. First, I think you should spend some time playing with Ruby's method lookup and figure out how it works. When you include modules, they end up in a linear stack between you and your parent class, and you can use that to your advantage. Once you know that, because 'super' works, it goes to the parent class.
00:28:24.480
It's also worth knowing that if you extend an object with your module, then you’re putting those modules in front of it, between its Singleton class and its real class, so you can use that for overwriting things—like if you want to edit bang methods or whatever. Modules are also terrific at limiting the scope of magic. If you’re going to do really scary things, then tuck it away in a module so you know that at least your scary things are limited to that one space.
00:29:12.600
You can control it. Remember, modules can modify individual objects. This one's really important because we don't see nearly enough of that. Pagination is a great example! We always have WillPaginate, and they build a collection class, but essentially it creates an array plus current page, next page, and another method—like 'per_page.'
00:29:29.000
Three methods on array—they could just return an array extended with a pagination module that has those three methods in it. It's still an array! So, I think we should try to replace inheritance with 'extend' and use modules more often in the inheritance chain. Less!
00:30:27.000
That's it! Thank you, guys, and are there any questions? Yes, Jeremy?
00:31:28.960
If you later include a module in your module that was already included somewhere else, you don't have the previous inclusion. So, I was wondering: when you include it for the second time, do you get the new ancestors? Does anybody know? I would have to check, but, it seems to get skipped and doesn’t modify your inheritance hierarchy at all, then. That’s actually safe, so that’s awesome because then you can keep stacking them in there, right? It doesn't matter if it’s happened or not.
00:32:02.480
Yes, right! So there is a case if you have A, then your module, and then B and your module again, but you call super from your module; you're going all the way up to A—you’re not going to stop at B. There are some intricacies in this, and this gets into a little scary module land, probably.
00:32:52.920
But yeah... okay, good point. The included hook gets run twice, so if you have something in the included hook, it gets run again. Be careful that code doesn’t edit your object again. Anyone else? Other questions? Okay, that's it! Thanks.
Explore all talks recorded at LoneStarRuby Conf 2009
+14