In Defense of Ruby Metaprogramming

by Noel Rappin

The video "In Defense of Ruby Metaprogramming" features speaker Noel Rappin at RubyConf 2022 and explores the merits of Ruby's metaprogramming capabilities. Rappin argues against the common view that metaprogramming is dangerous or overly complex, positing that it is an essential and enjoyable feature of Ruby that can enhance code structure and reduce redundancy.

Key points discussed in the video include:

Overall, Rappin’s talk aims to demystify metaprogramming in Ruby and inspire developers to leverage its strengths safely and effectively.

00:00:00 Ready for takeoff.

00:00:16 I was just telling Craig that the first time I ever gave a public talk was at an academic conference when I was a grad student. I was in the last session or the last day in a room that was this big, or possibly bigger, and exactly three people showed up. So, thank you for being more than three people.

00:00:22 Thank you for spending your time here this week. I really appreciate everybody coming out to be a part of the Ruby community. So, thank you! Let's start by looking at some code. Ruby fans here? Let's look at some code. This is real code from a fake project. It's a side project that I do; it's a project tracker. If you follow my newsletter or blog, you may have seen me write about it.

00:00:43 The important point here is that this project tracker has cards. Every card has a status, and the status has been abstracted out to its own object. We have a parent abstract class called Status and a series of subclasses that correspond to the individual status types: done, started, unstarted, and so forth.

00:01:07 Every client of this tracker deals with the abstract class. Obviously, we have other functionality in these classes as well; this is just a simplified schematic of the problem. The parent class defines various services, including creation methods for each individual status, strings for each status, predicate methods, and the main mechanism by which we convert from the Active Record data structure to our status.

00:01:40 Now, the main concern here is: how does this code make you feel? When you look at it, does it seem good or right, or does it give you that feeling reminiscent of unmatched parentheses, the kind that ruins your day and prompts you to fix it immediately? The first image I often use is just a bunch of text.

00:01:59 To put this in a more domain-specific context, does this code have duplication? In one sense, the answer is clearly no. If I want to find out if a status is done or not, there is one, and only one, place in the code that has that information. But on another level, it feels like there is duplication. What happens if the list of statuses changes or if we need to add a new status, such as 'deployed'? What would that entail?

00:02:30 This talk is titled 'In Defense of Metaprogramming'. My name is Noel Rappin, and I work as a staff engineer at Chime Financial. There are several Chime employees in the audience here who will be happy to talk to you about Chime or anything else. You can find me at noradman.com, and I promised myself I'd only get self-promotional if I finish this talk in under 30 minutes. We'll get there at the end.

00:03:09 I want to start with a definition to ensure we're all on the same page. There is a formal definition of metaprogramming that involves changing behavior at runtime. This is somewhat related to what we often think of in Ruby: the kinds of things we refer to as metaprogramming in the Ruby community. What we're really discussing here falls under the category of 'magic'.

00:03:40 My definition of magic is: any program behavior that is not normally specified the way someone from another language would expect. This includes things like class definitions, method definitions, typical method dispatch, etc. Anything beyond that we will call 'magic'. Ruby is designed to make magic easy.

00:04:06 We have things like 'define method' and 'method missing'. Both of these allow you to create functionality at runtime without using the standard 'def' keyword. We also have 'instance eval', which specifies the relationship between a message and its receiver at runtime. These features allow us to perform some pretty magical operations.

00:04:40 In the past, Rails contained long method missing hacks, such as 'find by email and name', where things were arbitrary. Once Ruby added keyword arguments, these hacks became unnecessary, leading to a cleaner approach. We should also remember that testing is essential, as it ensures correctness but also aids in discoverability.

00:05:03 While this talk highlights the strengths of metaprogramming, there is a tendency in the Ruby community to become defensive. For various reasons, it feels like we've pulled back from the magic that Ruby offers. This is evident in how far we've come since 2008, when the first speaker I heard, Dave Thomas, praised the flexibility and advantages of metaprogramming.

00:05:44 Dave asserted that the problems with metaprogramming are not technical; they're people problems, resolvable through training. Over the past 15 years, however, the Ruby community seems to have shifted towards viewing these issues as technical. This has led to resistance against metaprogramming, with some arguing it's too complicated or hard to maintain.

00:06:17 While it can be complicated and requires careful implementation, my argument is that we've overcorrected. We must embrace Ruby's capabilities and not shy away from its flexibility. I have grappled with how to present this during my talk—should I advocate for unfettered metaprogramming, pushing its use at every available opportunity, or should I remain reserved and cautious?

00:06:41 I've felt this tension and uncertainty throughout my preparation, and I want to explore where Ruby excels in metaprogramming and how it can be effectively utilized. Each instance I've witnessed raises questions about the function's real value.

00:07:04 Ruby handles certain types of complexity exceptionally well. Revisiting that earlier code, it is not structurally complex and is easy to grasp if you observe it line by line. However, the organizational complexity becomes apparent when you consider various related concepts that are embedded in the code without enforcing their relationships. This lack of enforcement increases the risk for inconsistencies.

00:08:03 For instance, if I wanted to add a new status called 'deployed', nothing stops me from creating inconsistencies between the class name, the string representation, and the predicate methods. Over time, these variances accumulate to create confusion that makes the code harder to work with, undermining its structural simplicity.

00:08:40 The surface area of the domain exposed by such a codebase proves to be a hindrance over time. To address this issue, using metaprogramming provides a mechanism to consolidate related functionalities into a cohesive framework. Let's explore a different methodology, maintaining the use of subclasses, but also implementing a more uniform approach that leverages Active Support's constantize method.

00:09:07 By defining a list of statuses and looping through it to dynamically create the necessary methods, we can maintain clarity in the functionality while reducing redundancy. This approach retains the flexibility of Ruby's features without compromising on the organization and manageability of the code.

00:09:58 We could also enhance this method by implementing Ruby's inherited hook, which allows us to simplify the addition of new statuses with minimal manual setups. By employing this method, we ensure that whenever a subclass is defined, the necessary methods are effortlessly created, facilitating future adjustments while minimizing the potential for errors.

00:10:40 Realistically, although this code is more complex than the original, it offers better separation between data and logic. The newfound clarity aligns the distinct responsibilities of structural data (status type) and logical behavior (method operations). Adopting metaprogramming techniques like these has clear benefits, with the potential for greater consistency and accuracy in our implementations.

00:11:12 I hope I have effectively demonstrated a reasonable use of metaprogramming. If not, allow me to address how one might persuade skeptical colleagues that this approach is indeed a valid strategy within a codebase. Firstly, it is crucial to arrive at discussions with humility, recognizing the validity of their concerns and addressing them directly.

00:11:55 The questions that arise include the correctness of the implementation, the readability, discoverability, and compatibility of the code. To effectively influence change, it’s important to talk honestly about the trade-offs and potential concerns involved with metaprogramming practices.

00:12:34 A specific argument is that if the structure derives from external data, like Active Record, that suggests it could be a good candidate for metaprogramming. Magento and Rails have their inherent forms of metaprogramming—whether that pertains to XML structures or URL patterns—so these considerations underscore the applicability of metaprogramming.

00:13:09 Testing remains a paramount aspect of this conversation. Not only does it help confirm correctness, but it can also illustrate proper usage to newcomers. Pointing to tests as examples of functionality can alleviate concerns about discoverability.

00:13:44 It's crucial to document thoroughly when implementing metaprogramming because, unbeknownst to many, understanding these abstractions can be challenging, especially for those less familiar with Ruby's conventions. When utilizing methods like 'define method', incorporate sample versions within comments to clarify intentions and functionalities.

00:14:22 Given the intricacies of using metaprogramming, allow for ample chance of discovery and understanding. This will forge a stronger synergy between experienced developers and new additions to the team. Thus, it’s crucial to become the local expert and provide guidance through pairing and collaborative discussions.

00:15:02 Embracing this fun aspect of coding doesn't just apply to traditional methods or familiar patterns. The ability to write Ruby code that employs metaprogramming techniques encourages creativity and innovation, but done too excessively might lead one astray.

00:15:43 In conclusion, let Ruby excel at what it does best. Metaprogramming serves to greatly alleviate specific complexities, particularly when the structure of the code harmonizes with its data. Stricter does not always equate to better; sometimes it is merely stricter for the sake of being so.

00:17:02 I see I have been speaking at great pace, covering more content in a shorter time than anticipated. Before I wrap up, I want to share that I am currently working on the fifth version of the 'Pickaxe' book, Programming Ruby, which is available for pre-order. It's about two-thirds complete, and you can find it via the link provided.

00:17:44 The coupon code will remain active until mid-December, so there’s no need to rush. However, I am certain that a print copy will be available around April next year. Additionally, you can find me on platforms such as Twitter or Mastodon. I work at Chime Financial, and I extend my gratitude to them for supporting both the conference and my visit here.

00:18:10 Thank you all for coming out and being a part of this community, and my sincerest appreciation goes to the organizers and everyone who has worked hard to make this happen. Your time here is greatly valued.