Crafting elegant code with Ruby DSLs

Ruby

Tom de Bruijn

@tombruijn

#ruby

#domain-specific-language-dsl

#meta-programming

#code-quality

#software-architecture

#rubygems

#developer-experience-dx

Crafting elegant code with Ruby DSLs

by Tom de Bruijn

In the video 'Crafting Elegant Code with Ruby DSLs,' Tom de Bruijn presents a comprehensive exploration of Domain-Specific Languages (DSLs) in Ruby programming, especially focusing on their elegance and maintainability.

Key Points Discussed:
- Tom shares his passion for Ruby and meta programming, expressing how they facilitate elegant coding that helps solve complex problems without hindrance.
- He introduces the concept of DSLs, explaining their role as tailored code structures that bring clarity and ease to developers, using examples like Rake, which simplifies command-line configurations.
- The advantages of using DSLs include:
- Improved readability and maintainability of code.
- Reduction in the use of less developer-friendly formats like YAML.
- Better accessibility for non-developers to understand the code structure.
- Tom acknowledges the criticism that DSLs often receive, citing experiences where developers shy away from gems with DSLs due to perceived complexity.
- To illustrate practical usage, he dives into crafting a DSL to transition from complex hash structures to method calls for defining attributes in a Diagnose report.
- The presentation highlights meta programming techniques, such as dynamically defining methods using 'define_method' and enhancing the readability of code through well-formed DSLs.
- Tom discusses the development of an extended DSL to introduce plugins, granting flexibility for users to customize functionalities without altering the core codebase, thus improving maintainability and developer experience.
- He emphasizes that meta programming, though daunting, can enhance code readability and should be viewed as a valuable tool in a developer's arsenal.
- The session concludes with an encouragement for developers to embrace and explore the use of DSLs and meta programming strategies for crafting elegant and maintainable Ruby code.

In summary, attendees are left with valuable insights into how DSLs can streamline coding practices in Ruby and why they are worthwhile despite their mixed perceptions.

00:00:11.719 All right, welcome everyone that stays. I am here today because I love Ruby. Who else is here because they love Ruby? Great! Otherwise, you're in the wrong place. I love Ruby because I can write really elegant code that makes the developer in me happy, which I think was the intended purpose. It doesn't get in my way, allowing me to focus on the problem at hand.

00:00:20.240 More controversially, I love meta programming, but I've been burned by the meta programming flames before, so I know what not to do. My name is Tom de Bruijn, but I go by Tom BR on most platforms. I work at a company called AppSignal. We are an APM that collects all kinds of data like errors, performance metrics, logs, uptime, and more.

00:00:34.320 It all started with just a Ruby gem that people installed in their apps. When you install it and load it in, it automatically starts instrumenting things like your Rails app, your Sidekiq code, your Sinatra, your Hanami, or your Delayed Job, whatever you run; it just works automatically and you don't have to do anything. This is only possible because of meta programming. It should just work without setup, but as with everything, complex systems become very complicated in all the different kinds of ways they are used.

00:01:14.200 Since we ship a Ruby gem to be run in someone else's app, you have no idea what kind of systems the other people are running. There are all kinds of unique and weird setups that I've seen over the years, and they can be difficult to debug sometimes. We have a tool called the Diagnose Report which you're seeing here. When people have a problem, they can run a tool that collects more than 100 data points in 12 reports and sends that data as a big JSON blob to our server. Then our server runs validations, and you can see it in a somewhat nicer visual UI than just a JSON blob.

00:01:56.240 However, there's a problem with this: it's an unmaintainable mess. I can say that because I wrote it, and no one else wants to touch it. I'm the sole responsible person for this code, so I'm not hurting anyone's feelings by saying this, but they are hurting mine a little bit because they complain about the code, which is fair. It looks something like this: I just grabbed a very small snippet; it's much larger than this. What I want to zoom in on is a particular piece of code that defines a data point on the report. There are hundreds of them. There are some magic keys for the hash that you have to know exist, and if you make a typo, good luck.

00:02:46.080 There's some kind of validation going on, but I honestly forgot what the highlight does, so there's some custom validation you could do here if you pass in a block the right way. Really, if I, the creator, have trouble understanding this, it has to change. Today we are going to look at crafting elegant code with Ruby DSLs. Let's start off with some definitions, because if you're like me, there are only so many abbreviations that fit in your brain, and they don't really capture my imagination.

00:03:34.040 What DSL stands for is domain-specific language. Some other people have spoiled this conference already, but even that is not entirely clear to me. What are we talking about here? What domain are we talking about? What do I type in my browser? What TLD do I use? Some domains you may know are actually part of Ruby, like this could be considered a DSL; it's a way to quickly define some getter and setter methods. But Rake, one of the most popular gems we have, has a DSL to define how the command-line tool you are making works, and it does all the logic for you. You can just run it from the command line, and it knows how all of that ties together.

00:04:44.080 Your app may also already have some kind of DSL, either explicitly or implicitly, because a DSL is a collection of terminology about the problem domain that it is solving. There is some kind of preference for how you use your code: with blocks, hashes, or all kinds of things. So why would you want to use a DSL? For one, it's kind of like an API; it's not all that different, but I would consider it to be friendlier. At least that's the idea.

00:05:09.239 If I bring out the inner Java developer in me, which I started way back when, and I write something for Glimmer, which you can see on the right, it creates a slightly more standard API kind of experience. I know which one speaks to me more, which one I could easily go back to and change. The DSL at a glance gives me a view of how the hierarchy of these components work and what kind of attributes or events are attached to them. Most gems start off with DSLs because they have some kind of configuration going on, like Rails; you don't want people to fork Rails just to change the time zone, so there’s a DSL for the config to set a time zone. There's also a DSL on the various last lines to add your own config options; you don't have to fork Rails to add your config options.

00:06:31.800 Most importantly, I don't have to write YAML. I don't know about you, but I already write too much YAML in my daily life. Too much Docker, Docker Compose, and all kinds of other tools. So if I can write it in Ruby, that just makes me much happier because that's the language I want to write in. It also allows me to write other pieces of code and generate that code for me. If I had to write a 'CREATE TABLE' statement to save my life, I wouldn’t know how to do it—it's been years! I'm really happy that Rails does this for me, and it does this for dropping tables as well. I don’t really have to think about it; it’s all handled by the DSL.

00:07:29.520 Maybe most importantly for this talk though, a DSL can help make high-turn code more maintainable. If I were to implement Rake myself, I might end up with a long case statement that I have to revisit every time I want to change something in the CLI, which isn't desirable. A DSL should also make it easier for non-developers. There was a client sitting next to Stephen on the first day who was reading their RSpec code and thought, 'Oh, that's just the notes!' No, that was the actual code. So if non-developers understand a DSL, maybe that indicates that my colleagues, who aren't Ruby programmers, can also modify this DSL if they want.

00:08:01.159 But then, are all DSLs good? That's maybe the sales pitch I'm making here today. Well, they should make reading and writing code easier. I'm not saying they shouldn't make it very easy that no one has to do any complicated code writing at all, but it should make reading the code easier, which is what we do most of the time. However, there are definitely people that very much dislike DSLs. I've seen and been a part of discussions where people say, 'Oh cool, gem, I really want to use this gem to solve my problem,' and then they find out it has a DSL and say, 'Oh, I am going to use something else.' This feels really weird to me because dismissing something outright just because it has a DSL doesn't help you solve your problem in the way you want.

00:09:38.000 You could also look at the documentation to see how it works. The example many people make when they talk about DSLs, and maybe too much magic in Ruby, are RSpec and Rails. People have complained a lot in the past, saying, 'Don't use Rails, it's too much magic; I don't know what's going on.' The same goes for RSpec: people ask, 'What is the difference between let and let!?' I must admit that sometimes I forget which one to use, but at some point I learned and now it's burned into my brain. That being said, I love RSpec; it's my favorite go-to testing framework! I’ve tried other ones, but unfortunately, Minitest is not for me—even though they do have an RSpec syntax, so it's the best of both worlds.

00:10:49.679 But still, RSpec, even with its DSL that people complain about, I really love it; it's great! Today, we're here to craft our own DSL. As I showed you before, this code is not horrible, but it makes me cry a little every time I have to change something. It's just not the way I would want to define this stuff. So today we're going to look at a domain DSL for this specific domain—the Diagnose report representation—so that I can create a nice HTML for it and make it easier to maintain.

00:12:02.200 What I've settled on is this DSL that does the same thing as the hash we just saw, but instead of a lot of hashes and syntax around that, it just works with method calls. The first thing you probably want to use when you have some kind of data structure to represent in a Ruby object is accessors. We know how to do this: you just define accessors on the class, and now you have all these methods available to set the values on the class.

00:14:03.560 However, I want to iterate over all those attributes set on that class without having to call them all manually. While I may have made my class easier to use, my view is becoming very lengthy because I have to call everything separately. I want to have these methods, but this doesn't work well with Ruby's attribute accessors.

00:14:50.120 So instead, I’m going to write our own methods. But now, for every data point, I would have to write those methods to maintain, which creates a new problem of having a lot more code to maintain. This doesn’t fix my problem. Instead, what we're going to do is dynamically define these methods. We'll first create the 'attribute' method on a report class; it receives the name and the metadata of the attributes. This is all understandable but now we need to introduce some meta programming.

00:15:30.160 We’re going to define a method using the 'define_method' method. This is where the meta programming kicks in. 'Define_method' works similarly to just normally defining a method; it takes the name of the method you want to define and a block which becomes the method body. Whenever you call the method, you call that block. If you want to create a writer, just add an equal sign at the end, and Ruby will handle the syntax magic there.

00:16:49.680 We will merge all the metadata with the value so that they're all in one place. Now whenever I want to set values or call them, I can use the 'attributes' methods to return them all. This all works, but I'm still not happy. This is great but I find there's too much going on with hashes. I have a thing against hashes for this kind of approach, so instead, I want to use blocks. Blocks are my favorite feature in Ruby. They're versatile; we use them to iterate over stuff and to configure things.

00:17:58.000 Instead of iterating over, we're just going to pass a block to our attribute method, which we can call using the 'yield' keyword in the method it is defined in. The argument passed to yield will be the attribute DSL instance, and the block will receive arguments, allowing you to call methods on that argument. So far, so good. Very quickly, I've introduced the concept of a DSL class, which is a neat way to separate concerns in your codebase.

00:18:34.400 You can just create a class specifically for the DSL using the same tools we've defined methods on before. Now you've got a class just used for the DSL that you can discard later, and it won't interfere with anything else. Puma uses this concept with their config file, which knows that methods like 'worker' and 'preload app' exist. They throw all that code into a file via their DSL class, and by the end, they just ask for the options hash from it.

00:19:10.720 Going back to blocks, we can call a block; this is where my personal preference kicks in. I don’t want to keep calling 'attribute' every time I want to set a label, a validation, or any kind of functionality. So can we omit it? Yes, we can! If we use 'instance_eval', we can change how the block calls, rather than the place it was defined. We can pass the block along to the object in which we want to evaluate the block, which in this case is the DSL instance.

00:20:58.880 Whenever you look at what 'self' is in that block, we've changed it to the DSL instance instead of the report config. Now that we're in that block, we can call 'label' as a method on the DSL instance. This is effectively just an abstraction away in a nicer DSL.

00:21:46.760 So which one would you use? Do you have a personal preference yet? It's up to you to decide as the creator of the DSL which you prefer. Personally, I lean toward 'instance_eval', but not always. If I add some kind of validation to my DSL, I prefer to be explicit about what I’m checking. If I’m checking something or setting an error, the context can change depending on the check, so I want to be clear about what methods I’m calling.

00:22:50.080 I don't want to call these blocks immediately because they're for the parsing logic we run when the JSON arrives at the app, so I want to store the blocks in an array for later. Whenever I do want to call them, I just call on the blocks I’ve stored. So far, so good, we are at the halfway point. We’ve already discussed a lot. We know how to make this DSL. I'm happy to be able to make our current code base significantly better!

00:23:58.120 If you are an aspiring gem author and want to be cool like other gems, you can make something like this work; you have all the tools you need. However, I must tell you one thing: you need to implement all the other pieces. This is just about the DSL. Now, let’s look at more complicated things for our specific domain. I mentioned I have about 12 reports, all with different kinds of metadata for every attribute.

00:25:46.080 For example, I have a language report that contains stuff like what version of Ruby it is—MRI, JRuby, Rubinus, etc. But there’s also other information about other languages like OTP version, which is Erlang—if anyone knows what that is. I don’t want that to show up whenever someone looks at a Ruby report. It confuses everyone, and I want to avoid that.

00:27:10.160 If I have this kind of shared logic between different classes without making them identical, I would use modules. To recap, there are two main ways to use modules: we can extend the module in a class to add a bunch of class methods defined in the module or include them at the instance level.

00:27:50.400 We are going to create attribute classes for our language report, extend the metadata module, and define what kind of metadata this report contains. However, I don’t like this approach because it puts a lot of responsibility on the user to make these classes include the module and define what kind of metadata is there. This can easily break, and while functional, it's not ideal.

00:28:38.080 So instead, we're going to create a DSL to extend our DSL. If you like meta programming, this is it! We’ll create a label class plugin that extends a kind of plugin class, defining what kind of metadata that plugin allows. Then, on the report, we can include that plugin, making the metadata available on the attributes. You may think, 'Wow, that's a bunch of over-engineering for something so simple,' and yes, that's true.

00:30:03.760 Plugging in allows for great flexibility; you can add a variety of functionalities to a gem without needing to solidly integrate them through the core codebase. Users can customize their reports to fit their exact needs, enhancing maintainability and reducing the workload on the gem creator. As a result, I end up making a gem that I can simplify and walk away from, while the user has the tools necessary to expand its functionality.

00:31:20.960 In conclusion, thank you for listening! I hope you learned why you may want to use a DSL and picked up a few meta programming techniques today. Remember, meta programming is not something to be feared—it is simply another tool in your programming toolbox, and it can be readable and maintainable.

EuRuKo 2023