My Meta Moments

by John DeWyze

In the video titled "My Meta Moments" presented by John DeWyze at RubyConf 2016, the main focus is on the process of learning meta-programming in Ruby. DeWyze shares his journey with meta-programming, highlighting its allure as a method of writing code that generates other code, while also emphasizing the challenges and complexities involved.

Key points discussed include:

- Definition of Meta-programming: DeWyze defines meta-programming as the ability to write code that can generate or alter other code at runtime or compile time, offering efficiency and flexibility.
- Perils of Meta-programming: While meta-programming can be powerful, DeWyze warns of its potential pitfalls, such as unexpected behaviors, complicating testing, and performance issues. Clever code can become hard to maintain, and simpler solutions are often preferred.

- Benefits of Meta-programming: Despite the challenges, meta-programming can provide efficiencies, especially in dynamically creating methods and behaviors. It can be enjoyable and should encourage experimentation, especially for beginners.

- The 'put-er' Gem: DeWyze introduces his debugging tool, 'put-er', which allows developers to insert tracking statements into their code to monitor behavior during runtime, showcasing practical applications of meta-programming.

- Meta-programming Techniques: He explores various techniques, including using send, method_missing, and define_method, discussing their applications and implications in Ruby programming.

- Understanding Inheritance and Composition: The presenter elaborates on how Ruby handles inheritance and composition, illustrating this with examples that clarify the roles of include, extend, and ancestors.

- Singleton Classes and instance_eval: These concepts further enhance the understanding of Ruby's flexibility, showcasing how instance-specific methods and context manipulation can be achieved.

- Resource Recommendations: DeWyze concludes with recommendations for further reading on meta-programming, including resources like "Metaprogramming Ruby 2" and online articles that helped shape his own understanding.

Overall, the video emphasizes that while meta-programming in Ruby offers significant opportunities for power and flexibility, developers should be mindful of its complexities and potential abuses. DeWyze encourages attendees to experiment and learn to enhance their programming skills.

00:00:15.139 Hi everyone! I know it’s a little early, but I want to give some people time to join. I have the deck up, and you can download it if you want to follow along. My name is John DeWyze, and this talk is titled 'My Meta Moments'. It focuses on my process of learning meta-programming, the project I worked on, and the concepts I needed to understand in order to effectively use it.

00:00:26.820 I’m going to jump to the bottom of this slide first. I have several code snippets that we will explore together. If you want to see them in their entirety later, you can download the PDF at speakerdeck.com/dewyze/meta. I've been a software developer at Braintree in Chicago for about two years, and this talk reflects my desire to learn more about meta-programming.

00:00:44.940 You can check out my GitHub profile at dewyze if you'd like. Before we get started, I want to include a required disclaimer: the views expressed in this presentation are my own and not those of PayPal or its affiliates, including Braintree. Since this is my first time speaking at a tech conference, I would really appreciate your feedback. I hope I speak slowly enough and explain the topics clearly, without making too many assumptions about what everyone knows.

00:01:06.510 I strive to use language that is inclusive and welcoming, and most importantly, I hope you gain something valuable from this talk. I’m open to both compliments and constructive criticism, so if you’re willing to share, please find me afterwards. It’s not exactly anonymous feedback, but if you hide your name tag, I won’t know who you are.

00:01:33.630 Let’s dive into the topic—what is meta-programming? I thought about using the Wikipedia definition, but I find it boring and too lengthy for our purposes. Instead, I'd like to offer my perspective, which may not be official but is certainly useful when learning. First off, it’s writing code that writes code. It sounds almost lazy, but it means you can create new behavior that can be added at runtime or compile time.

00:02:03.390 For example, if you’ve ever used Factory Girl for creating objects in testing, you’ll find it’s actually creating methods dynamically based on the parameters you provide. Another aspect of meta-programming is that it allows you to write code with incomplete information. For instance, Rails relies on magic methods, such as `find_by`, which operate with minimal information. We'll discuss this more later.

00:02:43.380 Finally, meta-programming enables you to write code very efficiently. For example, if we have a user object, we can retrieve all methods from it, look for those that begin with 'validate', and call them dynamically using the `send` method. This concise three-line approach can invoke a lot of methods without needing to know them in advance.

00:03:09.780 Before we go further, I want to address some of the perils of meta-programming. When I told someone on the plane that I was giving a talk on this topic, they asked if I would just tell people to avoid it. Not quite! There are some caveats to consider. Meta-programming can produce unexpected behaviors, and if you aren't aware of certain method calls, you might unintentionally invoke a method that should only run under specific circumstances.

00:03:52.830 It can also complicate testing. Since meta-programming might manipulate the way code operates, such as modifying how inheritance works, it can sometimes make testing messy without resorting to cumbersome stubs and mocks. Additionally, it can slow things down. When new methods are added, they can invalidate method caches that improve performance, leading to increased complexity in managing your code.

00:04:52.960 While meta-programming can be clever, it often requires knowledge of surrounding code and unexpected interactions. Clever code can quickly become hard to maintain, increasing the burden on anyone trying to understand your codebase. Often, the clearer, more straightforward solution—even if it takes more lines of code—is the better choice.

00:05:14.050 That said, there are some perks to meta-programming. It's efficient! For example, when we refer to that validate method, I can call hundreds of methods with just three lines of code without any concern. It's also powerful, allowing you to change class behaviors and how classes inherit from each other. Perhaps most importantly, it’s fun! If you’re new to programming, when someone tells you not to meta-program, you should experiment with it. Just avoid using it in production code.

00:05:51.460 Ruby facilitates a lot of exciting features, and the curiosity to learn about meta-programming should be encouraged. To illustrate, I think about Uncle Ben's words from Spider-Man: 'With great power comes great responsibility.' When I was a kid, I probably pretended a cardboard tube was a lightsaber, and I soon learned that they could be dangerous. However, if used correctly, then you can be like a Jedi!

00:06:40.230 Now let me introduce something I call 'put-er'. It's often mispronounced, but that’s beside the point. I’ll explain what this gem does and how the meta-programming tools I learned were applied to build it. It’s a bit of shameless self-promotion, but I believe it's easier to understand concepts through examples within a concrete idea than by just discussing them abstractly.

00:07:03.070 So, 'put-er' is a puts debugging gem—a fancy method where you insert puts statements throughout your code to track what’s happening. You can find it at github.com/dewyze/put-er. This is a little script that utilizes 'put-er': we create a user class, make the username readable and writable, and initialize it with a number, forming a username based on that number.

00:07:34.750 We also define two methods, `uppercase` and `lowercase`, which are basic wrappers around Ruby's built-in functionality. Note that I often use semicolons in my code to keep lines shorter, but it does work properly. After that, we call `put-er.watch`, with APIs for 'watch' and 'follow'—we'll discuss both later.

00:08:02.990 We create an array of users, ranging from zero to four, making a total of five users. We’ll add each user into the array and call `uppercase` on the first user. Next, we’ll iterate over the array to call `lowercase` on the even-numbered users (0, 2, and 4). With `put-er`, you'll see some snippets of what happens.

00:08:30.920 In the user class, at line 15 of 'watch', I call the new method and pass in zero, resulting in a user. The following line captures the first instance of the user, and when I call `uppercase` with no arguments, it capitalizes 'user_0'. From the last user instance created, which corresponds to index four, I call `lowercase`.

00:09:12.280 One exciting feature of 'put-er' is how it helps with integration tests or understanding a new codebase that you haven't seen before. You can watch a class and see every place where it’s called, including any class methods or instance methods invoked and the arguments passed.

00:09:41.500 This functionality is helpful, especially for examining what's happening under the hood of Rails. For instance, if you’re looking into Active Record, which calls ID and class often, I’ve added options for ignoring these methods, allowing for a clearer view of the interactions.

00:10:02.690 Now, let’s cover the topics we’ll discuss regarding the tools used—some are more strictly classified as meta-programming than others, yet they are all useful to know. There are additional methods of meta-programming that I won't be addressing. Also, I encourage you to explore edge cases in meta-programming, as there are many intricacies, such as how 'self' changes depending on the context.

00:10:29.780 First up is the `send` method, a powerful tool allowing you to call any method by invoking it with a symbol or string representing its name. This means you can access private methods since they can be invoked through 'send'. Let’s explore how this works using the 'prai' gem, similar to IRB, for practical examples.

00:10:53.050 We define a method called `send_me`, which notifies us when it is sent. Creating a new instance of our sender and using `send` to call `send_me`, we will see that it was invoked successfully. This demonstrates how `send` can be used to call methods dynamically.

00:11:21.150 Next, let's discuss inheritance. Most of us are familiar with how inheritance works in Ruby—similar to vehicles where a truck behaves as a type of vehicle, so does a user inherit behaviors from Active Record. Users have methods like `save` or `update_attributes` that come from Active Record.

00:11:53.160 However, what’s interesting is that when you use `include` and `extend`, you're actually altering inheritance. For example, if a television includes `watchable`, it is inheriting from that module. This realization changed my understanding of composition and inheritance.

00:12:20.130 To explore this further, we can use the `ancestors` method to examine the inheritance chain and understand the order. Virtually all objects inherit from `Object`. You'll notice that `Kernel`—where `puts`, `chomp`, and `eval` methods originate—gets included, followed by `BasicObject`.

00:12:51.250 Instance eval plays a significant part here too. Now, I’ll touch on `prepend`. While it might not seem specifically like meta-programming, `prepend` allows you to expose module methods before methods defined in the class. You might consider it a handy tool for creating proxy modules.

00:13:11.620 Let’s set up a test class, apply `prepend`, and ensure our proxy methods function correctly. This process enables creation of methods that intercept calls and manipulate behaviors before delegating to the original methods. By looking at the ancestors of our test class, we can see how the proxy sits within the inheritance chain.

00:13:48.290 Now let's address method dispatch—how Ruby determines which method to call. It does so by searching through the ancestors, returning the first instance of the method it encounters. Visualizing this can be beneficial, but in brief, the position of methods in terms of inclusion impacts behavior drastically.

00:14:13.210 A practical example involves loan ownership in a class model, where we can define which loan owns what. By owning classes that include each other, we can see how Ruby prioritizes the method calls based on the ancestry. Now this information is critical, as it lays the foundation for understanding what follows.

00:14:42.470 So, let’s dive into `method_missing`. This method is triggered when an attempted method call cannot be found. It's quite useful in defining dynamic behavior: if we can't find a method, we can create it at runtime! By using a combination of `method_missing` and some conditions, we can tailor responses based on inputs.

00:15:21.840 For example, if a method matches a certain pattern, we can perform relevant action and return the necessary output. This approach gives us the flexibility to handle various situations dynamically, leading to a robust program that can adapt based on the context.

00:15:54.160 Now, let’s consider our own version of Active Record called lazy record. By incorporating `method_missing`, any direct calls to non-existent methods will trigger this construct originally defined to manage database lookups. By doing a simple pattern matching, we can provide results despite not having explicitly defined methods.

00:16:20.240 Next, we also want to discuss `define_method`, allowing the creation of methods that we need without explicitly writing them. This powerful approach can create custom methods at runtime in response to specific needs, enhancing flexibility when responding to user inputs or dynamic scenarios.

00:16:56.860 Continuing, let's talk about `BasicObject`. This is the parent of all objects in Ruby, and it has only a few methods, which makes it highly desirable for meta-programming tasks. It's also the source of the `method_missing` and `initialize` methods, allowing us to build the bare minimum functionality required for any object.

00:17:40.990 In Ruby, `singleton classes` are classes that are objects themselves. This interesting aspect of Ruby allows methods to act as instance methods of these singleton classes. They enable developing capabilities primarily to single instances of classes.

00:18:14.030 For instance, we might create a greeter class that defines a method to return greetings. By opening the singleton class, we can define additional methods seamlessly. Always exploring how classes and instances interact—as well as how singleton methods act—will help developers understand Ruby's unique approach to flexibility.

00:18:44.390 Following this, I’ll introduce `instance_eval` and `instance_exec`. 'Instance_eval' allows evaluation of code within the context of a particular instance, exercising greater control over internal workings. It offers mechanisms to manipulate and evaluate without needing traditional public access methods.

00:19:12.950 To highlight this, let’s form a spy class and examine how 'instance_eval' can access hidden instance variables like secrets. This presents a straightforward way to interact with otherwise private data in an object. On the other hand, `instance_exec` can accept parameters during a method call, expanding its use cases.

00:19:47.600 This duality of `instance_eval` and `instance_exec` reflects the power of Ruby's meta-programming capabilities. So, what did I do with my 'put-er' gem? It utilizes the concept of 'watch' and 'follow', allowing users to keep track of method calls while being flexible with how users can interact with objects.

00:20:24.650 The core idea allows a 'follower' object to observe an instance, meanwhile logging what happens as methods are called. This functionality is underpinned by incorporating method missing, allowing for tracking essentially all interactions while remaining non-intrusive to the original code.

00:21:04.730 Additionally, the `watch` feature encapsulates observing class-level behaviors. Here, we create an instance follower that can add custom details to interactions while storing a record of the instance’s original behavior. The fascinating aspect is that it amplifies the user's ability to understand how methods act in context.

00:21:48.820 Each of these tools and techniques aggregates functionality, enabling the perusal and tracking of behaviors across multiple instances while decorating behavior with pertinent information. This represents the best of both worlds: rich interaction for debugging and preserving original behaviors.

00:22:23.670 In summary, I want you to recognize that meta-programming can open up numerous possibilities for learning and exploring your Ruby code. It provides the power to contribute to projects like Rails or Factory Girl, and even 'put-er', which is my first open-source project. I welcome contributions!

00:22:58.260 If you’re interested in meta-programming, here are some valuable resources that helped shape my understanding: Thoughtbot's article on writing a DSL in Ruby, Ruby Under a Microscope which provides deep insight into the internal workings of Ruby, and specifically, 'Metaprogramming Ruby 2' which dives thoroughly into meta-programming concepts.

00:23:27.430 I genuinely hope you now have some valuable tools and concepts to start meta-hacking your own projects! Thank you for your time; I welcome your questions. I'll take a few now, but if anyone prefers to ask later, I'm happy to stay up front to chat one-on-one.

00:23:55.000 Let's start with a question on the difference between `instance_exec` and `instance_eval`. The notable difference is that `instance_exec` accepts parameters, allowing you to inject values into the context of the block. Meanwhile, `instance_eval` does not take a parameter, but allows running code within the object's context.

00:24:32.450 Regarding the singleton class, it doesn't relate directly to the Singleton design pattern, which ensures that a class has only one instance. Rather, a singleton class is a special class metaclass that holds methods specific to that particular object instance. This distinction is crucial for understanding Ruby's flexibility.

00:25:12.440 With that said, if anyone wants to ask additional questions, feel free to approach the stage. Thank you again and have a great RubyConf!