Metaprogramming
Ruby ate my DSL!
Ruby ate my DSL!
by Daniel AzumaIn the video 'Ruby ate my DSL!' presented at RubyConf 2019, Daniel Azuma discusses the challenges and intricacies of designing Domain-Specific Languages (DSLs) in Ruby. He emphasizes that while DSLs can simplify coding by introducing expressive and concise syntax, they uniquely interact with Ruby's underlying mechanisms, which can lead to unexpected issues.
Key points covered include:
- Introduction to DSLs: Azuma defines DSLs as methods that don’t require a receiver object for invocation. He highlights their usefulness in creating commands that resemble native Ruby syntax.
- Common Pitfalls with DSLs: He illustrates how conflicts can arise when DSLs utilize method names that overlap with Ruby standard library methods, citing a personal experience with a Sinatra application that led to a debugging challenge due to a method name conflict.
- Using Metaprogramming in Ruby: Azuma discusses how DSLs leverage Ruby's metaprogramming capabilities, e.g., the Rails framework uses instance_eval to dynamically change the context of a block. This allows DSLs to extend their functionality while still being rooted in Ruby.
- Control Over User Input: The speaker warns that DSL users often inject standard Ruby code into the DSL. Thus, designers must anticipate how end-users might misuse their DSL and incorporate defenses to prevent unexpected behaviors.
- Creating a Sample DSL: To solidify his points, Azuma teases the creation of an illustrative DSL called 'nano_spec,' aimed at showcasing Ruby DSL designs and their challenges.
The presentation underlines the importance of intentional design in DSLs to ensure both usability and safety when integrating Ruby features.
00:00:12.920
Before I get started, I just want to say that we're in Nashville, Tennessee this year. I'll be honest, I'm kind of a stuffy West Coast city dweller, and Nashville has not really been on my bucket list for places to visit. It was not on my radar, but since the conference is here this year, I decided to come a few days early. My wife and I came on Friday, and we got a chance to check out the city, some of the venues, and some of the music.
00:00:26.580
I have to admit, I was really blown away. This is an amazing place, and we really loved it. How many of you have had a chance to check out some of the places so far? Just a few? We're here for three days, which means several evenings. I really encourage you, if you get a chance, go out and explore the city. Walk into pretty much any music venue or jazz club; you won't regret it. You won't be disappointed—there are some amazing talents in this city.
00:00:48.600
Anyway, that's enough of a sermon from me. We're going to have a gripe session today, right? Yes, that's right! We're going to gripe about Ruby and DSLs (Domain-Specific Languages). For those of you who are just coming in or those in the back, please move forward. There are plenty of open seats, and there’s a lot of code on these slides, so it'd be good to get closer to see it.
00:01:31.790
Not long ago, I had to write a Sinatra app for my day job. This is kind of a simplified view of what the code was. Basically, its job was to invoke a shell command, capture its output, format that output as JSON, and send that as a response. So, it was a really simple, straightforward app. I figured, hey, I've done Sinatra before—what could possibly go wrong? I tested it locally just to make sure, and as it turns out, there was a bug.
00:02:04.890
Can anyone see the bug? Raise your hand if you think you know what the bug is. Okay, a few hands. If you have your hands up, I might call on you. For the rest of you, here's a hint: this is the stack trace that you get. Anyone see it? Who thinks they know what's going on? Okay, no one's hand is up now. What do you think is happening?
00:02:32.970
The invoke method is a symbol too, but inside the echo. Okay, so inside the echo, there needs to be a symbol for it to be executed—not quite, not quite. We'll just move on. The problem here is that this call turns out that Sinatra has an internal private method named invoke, so this line actually calls Sinatra's method, not my helper method. Of course, it's a bit tricky to see that, right? The calling conventions don't match up, and we get this obscure exception that's hard to parse. It's hard to know what's going on unless you dig into Sinatra's source and understand what the DSL is doing.
00:03:17.890
Now, it's not my point here to pick on Sinatra; this happens a lot in Ruby across many libraries, especially when DSLs are involved. So, we're going to spend this session griping about this. We're going to see why some ordinary Ruby code can sometimes make DSLs go wrong, and then we're going to study some techniques that you can use to help harden your DSL against problems like that.
00:04:18.910
Quick intro before I get too far: my name is Daniel Azuma. I've been a Rubyist since about 2005, right around the time that Rails 1.0 came out. I've done a variety of things, including some Rails startups. Currently, I work at Google Cloud as a Ruby engineer. We don't do a whole lot of Ruby at Google, but there are several of us. A bunch of us work there, and a lot of us act as consultants for Google's engineering teams. My role is to assist the engineering teams building Google Cloud to help them tailor their products for our needs as Ruby developers, ensuring that Google's cloud works well for Ruby developers.
00:05:54.490
So today, what we're going to do is discuss some Ruby practices around DSLs. I’m going to take a stab at defining DSL. Before I get started, the talk preceding this went through a lot of definitions. Definitions are important, especially when you're discussing these topics because, as the last speaker mentioned, Ruby sometimes has different concepts. The concepts in Ruby can work slightly differently than in other languages.
00:07:01.150
When I speak of DSL, I'm talking about 'bare methods' that are not part of core Ruby. So, what does that mean, 'bare methods'? It refers to methods that don’t need a receiver object to be called, such as methods like `puts` or `sleep`, which look like commands in Ruby. These methods are provided by a library; they’re not part of the standard library or the core of Ruby, but they’re part of some external or third-party library.
00:07:34.350
DSLs can be very useful, making for some expressive and concise code. Looking back at our Sinatra example, the `get` method there is called without a receiver object; it’s called at the top level, but it is being used as if it were part of core Ruby. However, it’s not—it’s part of the Sinatra DSL. Now, how does Sinatra accomplish something like this? Sinatra adds this method to the main object. The main object is considered to be `self` when you’re at the top level of a Ruby file—every Ruby application has one.
00:08:52.180
Here's another example of a DSL, this one from Rails. Who's a Rails developer here? Okay, a lot of us. Many of us came to Rails through Ruby; that was my story. I know some of us like to criticize Rails a bit because it’s big and embattled, but really, it’s a nice solid framework. One of the reasons it works so well is due to how strategically it uses metaprogramming and DSLs internally to be expressive. Here’s an example from Rails routing that I copied directly from the Rails routing guide. We have methods here that are being called without a receiver, and those aren’t part of core Ruby but are methods defined on a mapper object in the router section.
00:09:57.840
When you call `routes.draw` in your routes file and pass it a block, that block gets executed using a Ruby construct called `instance_eval`. Within the block, what happens is that `self`, the current Ruby object, is set to another object—in this case, it gets set to a mapper object. Therefore, within the block, `self` is assigned to the mapper, giving you access to all that mapper's methods like `resource` and `resolve`. Setting `self` within the block or adding methods to an existing object are two useful techniques. A lot of DSLs essentially boil down to these two techniques.
00:10:48.890
They make it look like we're extending the language by extending Ruby, but despite that, it’s important to realize that it’s still Ruby. We’re still writing Ruby even though you have a DSL that might appear to extend the language and create new methods that look like part of core Ruby. So, your DSL users know this; they’ll still be writing Ruby even when they code to your DSL, and they’ll expect to define classes, define modules, call methods, and do all the things they expect from Ruby.
00:11:32.870
As a DSL writer or designer, it’s crucial that you plan for this. This doesn’t always come for free in Ruby, and you need to be intentional about making sure Ruby works well in your DSL. We’re going to look at a bunch of examples, and to illustrate things, we’re going to create a little strawman DSL. I’m going to call this `nano_spec`. Who here uses RSpec? A lot of us do, right?”},{
