00:01:32.280
There are some selfies being taken, but we really need to get started again, as we're on a tight schedule.
00:01:37.960
Remember, otherwise I will take away your lunch break. And everybody's like, 'What's lunch break?'
00:01:45.789
I need to take a picture with Mats. Okay, so we will have two talks, and they will be great.
00:01:58.920
Then there will be a lunch break, which means you need to go outside. I've also been told that people should come on time.
00:02:09.280
Schedule silence—I have no authority on this stage. Do I? No authority.
00:02:27.940
I love that! Yes, yes.
00:02:37.760
All right, we have to talk about the schedule. So, there will be a lunch break, as long as you cooperate with me.
00:02:44.360
If you don't, you know the consequences. I have been told there will be coffee and such, whatever 'such' means, but I'm guessing it includes food and snacks in the next break.
00:03:00.019
So, that's happening, and we need to do some technical stuff here. I'm just going to announce the next speaker right away to stay on schedule.
00:03:08.840
Breaks are for talking, while conference room attendance is for not talking and for listening. Again, thank you to my kind volunteers in the audience for the clapping, it works really well.
00:03:15.790
So, our next speaker is originally from Canada and now lives in Japan. He is the author of Mobility and an open-source contributor.
00:03:22.159
He is also very active in the Japanese Ruby community, and he will be teaching us all about metaprogramming, which is the machinery that drives Ruby.
00:03:29.030
We will be learning about writing modular, extensible code and tackling real hard problems. His goal is that we all walk away from here with a conceptual understanding to be more generalists.
00:03:36.290
Because, according to Chris, that's what we need more of in this community. So, give it up for Chris!
00:04:46.940
Look, okay, everybody, my name is Chris Salzberg. The title of my talk is 'Metaprogramming for Generalists.'
00:04:52.400
I apologize—it's the first morning of the first day, and we're starting with metaprogramming. But hey, don't worry, it'll be interesting!
00:04:57.950
Some quick points about me: my handle is 'ChillYama,' if you've seen that name around. I live in Tokyo, Japan.
00:05:05.810
The background picture is from my neighborhood. Don't be fooled; I'm not Japanese—in case that wasn't obvious. I'm Canadian, originally from Montreal.
00:05:11.660
I work at a company called Edgeago. We have a booth outside, so check us out.
00:05:18.080
In the open-source world, I am the author of a few gems, with Mobility being the most well-known, which is for translating model data.
00:05:24.230
I also write about things like the module builder pattern on my blog, Digimon.com. Okay, enough about me.
00:05:34.670
This is a two-part talk. In the first part, I'm going to present a new way of thinking about metaprogramming that I think will be more relevant and meaningful for solving generic problems.
00:05:40.250
In the second part, we're going to solve a generic problem and build what I'm going to refer to as generic software.
00:05:45.530
So, talking about generalist metaprogramming, it's Ruby's 25th birthday, and I think more than any other area, metaprogramming is where Ruby truly stands out among programming languages.
00:05:52.160
It's something that Ruby really embraces, and it underlies all the aspects of Ruby that Matz just mentioned—the human-like syntax, the DSLs, and the nice use of it.
00:06:00.220
However, we still have a hard time defining this term. Programmers, in general, struggle with this concept.
00:06:05.480
A basic definition states that if programming is writing code, then metaprogramming is writing code that writes code.
00:06:12.410
A second definition, from Wikipedia, describes it as a technique in which computer programs can treat other programs as data, also known as reflection.
00:06:19.130
It's a bit broader and also incorporates introspection, which is a way for a program to find out about itself or other programs.
00:06:26.000
By the end of the day, though, people can't seem to agree, and we tend to throw our hands up in the air and refer to it as a 'bag of tricks.'
00:06:31.520
This quote is from the book 'Metaprogramming Ruby,' which I highly recommend. What are those tricks? They include techniques such as monkey-patching, yield, self, and others.
00:06:37.250
In practice, however, we often use a different working definition that identifies certain Ruby methods as metaprogramming methods.
00:06:42.290
The classic one is 'evil', but also methods like 'define_method,' 'method_missing,' 'instance_eval,' and even 'send'—depending on where you draw your boundaries.
00:06:47.870
This is what I will call the 'what' view of metaprogramming. It describes these methods and how to use them.
00:06:53.510
However, I don’t want to focus on that today. If we zoom out on that view, we start to see a problem.
00:06:59.390
This is my concern: how we see metaprogramming as a community. We often view it as a niche, fringe topic.
00:07:05.099
It's often seen as scary, magical, or irrelevant to many of us. However, for another part of the community, which includes myself, this is not how we see it.
00:07:11.110
Metaprogramming is central to Ruby and what we do with it. I believe this divide exists because, as I said, the materials out there do not answer the fundamental question: What is metaprogramming for?
00:07:16.449
So that's what I want to tackle today. To do this, we're going to consider the role that metaprogramming plays in our ecosystem.
00:07:21.550
We start with fire. We put Ruby in the fire, and Matz is feeling the heat.
00:07:28.200
Down here, we have the Ruby committers building this language we love. Up here, we have the applications, where most of us spend our time.
00:07:35.219
We are building things that pay the bills for us, using Ruby. We understand how to use it, but we don't always know its internals or care about them—unless something goes wrong.
00:07:42.230
However, this picture is incomplete. I'm missing a layer here, and that layer is the libraries.
00:07:50.150
I will use the terms 'libraries' and 'gems' interchangeably here. Libraries fill the gaps that multiple applications encounter, which the programming language itself does not provide.
00:07:58.670
In Ruby, these libraries blend almost seamlessly into the programming language itself. ActiveSupport is a classic example of that.
00:08:05.360
So what does this have to do with metaprogramming? The point here is that metaprogramming overwhelmingly lives in this middle layer.
00:08:12.020
A classic example is ActiveRecord, where all your attribute methods and association methods are created using metaprogramming.
00:08:19.320
However, it is not just Rails or ActiveRecord; many gems use metaprogramming significantly. When you arrive at the application layer, you don't see all of this very clearly.
00:08:25.750
You might notice monkey-patching here and there, but you don’t often see a lot of metaprogramming in action.
00:08:32.610
This leads to an important question: Why?
00:08:39.050
Why is metaprogramming so prevalent in our libraries but virtually non-existent in our application code?
00:08:46.380
I think that this is an important question that hasn't really been addressed, and it might seem obvious, but the answer is more subtle.
00:08:53.720
To explore this, we need to consider another big word in my talk title: generalization.
00:09:00.650
Generalization is what libraries do; they generalize problems faced by more than one application into generic components that we can use in all our applications.
00:09:07.990
Here’s a definition from Wikipedia: Formulation of general concepts from specific instances by abstracting common properties.
00:09:14.620
We will dive into the details of abstracting common properties in the second part of this talk, but for now, I want to focus on this: What role does metaprogramming play in developing general concepts?
00:09:21.450
Because general concepts are what our libraries or gems implement for us.
00:09:28.720
To illustrate this, we need a general concept. I'm going to pick an example that I believe is relevant for everyone here: the concept of attributes.
00:09:36.440
To understand the role metaprogramming plays in the concept of attributes, let’s take a cue from genetics.
00:09:43.360
If there are any biologists in the room, I apologize in advance. In genetics, to understand what role a gene plays in a mouse's DNA, one technique is to conduct a knockout experiment.
00:09:50.490
You knock out the gene and then observe the differences between the mouse with the gene and the mouse without it.
00:09:57.890
So, let’s do the same with the concept of attributes by removing metaprogramming and observing the results.
00:10:05.150
This is our library example. The library gem has a single class called Model, which has an initializer that sets a blank hash of attributes.
00:10:12.720
It has a single class method called 'define_attribute,' which takes an attribute name and creates both a setter and a getter. The getter fetches the value for that key in the hash while the setter sets the value.
00:10:19.200
This is a very simple setup. Now, we’ll switch to the application perspective implementing this simple library.
00:10:25.030
We create a class called Talk, which inherits from Model, and we call 'define_attribute' twice for two attributes: title and abstract.
00:10:32.290
We create a talk, set the title, and retrieve it back. Similarly, we set the abstract and get it back. This is very simple stuff.
00:10:39.110
The key point here is that the abstraction is transparent. It's invisible. In our example, we use the word 'attribute' when defining the attributes, but once they are defined, we don't see that abstraction at work.
00:10:45.960
Now, let’s switch to the knockout case, which is without metaprogramming.
00:10:53.340
In this scenario, we have the same class Model; it initializes the same way with a blank hash, but we can no longer define methods dynamically.
00:10:59.050
So instead, the only thing we can do is define two instance methods: get_attribute and set_attribute.
00:11:06.150
They perform the same functions as before but require the name of the attribute to be passed in to fetch or set the value from the hash.
00:11:12.880
This forms our library example without metaprogramming. Now, if we check the application that implements this library, it again inherits from Model.
00:11:20.050
We still create a talk, but since we can't define the attributes dynamically, we allow users to set or get anything they want.
00:11:27.500
In this case, we create a talk and call 'set_attribute', passing the name of the attribute 'title' and its value, and we can get it back the same way.
00:11:34.380
I want to highlight that this example is really simple, but there are significant subtleties. The abstraction is now visible; the abstraction is literally the term 'attribute.'
00:11:41.970
When you want to set or get an attribute, you need to call 'set_attribute' or 'get_attribute'. The words 'set_attribute' and 'get_attribute' are right there.
00:11:48.910
As for the attribute names themselves, they become second-class citizens, as we're passing them as arguments to these methods.
00:11:55.940
Now, let's summarize the results of our little experiment between the application and library when comparing the cases with and without metaprogramming.
00:12:03.850
In our control case—with metaprogramming—in the application, the abstraction was invisible. We simply call attributes directly without thinking about the abstraction involved.
00:12:10.440
In this situation, we can say that the library speaks in the language of the domain, which is Talk, Title, and Abstract.
00:12:17.310
However, in the knockout case—without metaprogramming—the abstractions become visible. You have to specifically call out those attributes when interacting with them.
00:12:24.530
Here, the library speaks in the language of the abstraction, making it clear you have to work with the concept of attributes explicitly.
00:12:31.250
Now, turning to the library perspective, the comparison was simple. Our library was really basic in its implementation.
00:12:38.380
But in the knockout case without metaprogramming, the unknowns cannot refer to code; therefore, we must pass the name into the getter method, which then passes it to the hash.
00:12:45.540
In general, this makes the knockout case without metaprogramming more manageable to understand, given there is no hidden abstraction.
00:12:52.440
But if that's the case, why not eliminate metaprogramming entirely? Why would libraries and gems employ it when it arguably complicates things?
00:12:59.550
The context for this question isn't straightforward. You could implement something like ActiveRecord without metaprogramming, but it would look like a mess.
00:13:07.290
You would have to call 'get_attribute' and 'get_association' for every interaction, which is cumbersome and obscures the domain language.
00:13:14.520
For the application developer, these variables become known. You can define them without metaprogramming. However, for a library, they are essentially unknown.
00:13:21.250
This is where metaprogramming becomes invaluable for gem authors. It allows you to speak the language of the domain effectively.
00:13:27.800
Metaprogramming levels the playing field by enabling libraries to translate unknowns into code.
00:13:35.000
This perspective on metaprogramming is meaningful and actionable: it provides a working framework to understand it.
00:13:42.800
For instance, the unknowns at the library level deal with column names and tables, while the unknowns at the application level tend to be user data.
00:13:49.570
You, as an application developer, don't know what kind of user input will come through. You may not want to translate user input into code.
00:13:56.230
This clarifies the divide between applications and libraries: for libraries, metaprogramming serves as an enabler, while for applications, it can be viewed as an obstacle.
00:14:03.360
Moreover, we can see issues with our working definition of metaprogramming as a collection of methods.
00:14:10.000
Our assumption was that using any of these methods qualifies as metaprogramming, but I want to illustrate why this assumption can lead to misunderstandings.
00:14:16.890
Let's examine the use of 'eval': if we write 'eval foo,' our understanding of metaprogramming implies it is metaprogramming.
00:14:24.010
But what does this evaluate to? It simply calls the method for the variable 'foo' without translating unknowns into code.
00:14:31.100
'eval' may seem like metaprogramming, but it falls into a category of its own known as automatic programming.
00:14:37.560
Now consider the case when we append a string to 'foo': if we see 'foo = #{foo}', that still doesn’t change the situation.
00:14:44.150
The only way to make something unknown is to wrap it in a method; we name this method 'foo_gal'. It takes a string and appends it to foo.
00:14:51.920
This method still earns the title of metaprogramming as it is translating unknowns into code.
00:14:58.470
Using 'defined_writer' is similar to the second half of 'define_attribute.' Here, we create a method with 'defined_method'.
00:15:06.090
While its use of 'define_method' qualifies it as metaprogramming according to the original working definition, it does not actually translate unknowns into code.
00:15:12.650
Thus, we need a definition that differentiates the context of whether the attribute name is known or unknown.
00:15:20.125
For example, if we use 'auto_writer,' by passing a string or a symbol, those are generally known values.
00:15:27.240
In contrast, if we pass in an unknown value, it would accurately qualify as metaprogramming. So we can visualize these results in terms of our definitions.
00:15:34.510
The original 'what' definition encompasses numerous metaprogramming methods, and on the other side, the knockout does not.
00:15:41.140
As we introduce additional data points, we see conflicts, such as the eval example where it shouldn’t belong here.
00:15:48.300
Thus, we will establish a new definition grounded in the idea of 'how' instead of merely 'what'.
00:15:55.200
Therefore, we define metaprogramming based on its capacity to convert unknowns into code.
00:16:01.600
This revised definition will considerably enrich our understanding of metaprogramming.
00:16:09.720
Now we can explore how to apply this new understanding and how libraries can use metaprogramming to solve generic problems.
00:16:17.680
The second half of my talk is about generic software. This concept isn't my own; it comes from Jeremy Evans, the author of several gems including the ORM 'Sequel.'
00:16:25.660
In a talk from 2012 about the development of Sequel, he mentioned that one of the best ways to write flexible software is by writing generic software.
00:16:34.360
Instead of designing a single API that completely handles a specific case, you can create multiple APIs that handle smaller, more generic parts of that use case.
00:16:41.340
Then handling the entire case becomes simply a matter of gluing those pieces together.
00:16:47.840
Now, let's build an example to illustrate this concept—one about equality. In Ruby, there are several kinds of equality.
00:16:54.560
We will use double equals equality, which checks if two objects with similar content are equal.
00:17:01.630
For instance, two strings that have the same characters will be considered equal.
00:17:09.720
To provide an example, I didn't go to Sequel this time; instead, we are going to Rails.
00:17:17.780
After searching through Rails code, we found a test file with two classes intertwined with our topic.
00:17:24.900
The first class is called Address, where an 'add_reader' method takes street, city, and country.
00:17:32.360
In addition, there's an equals method that compares the street, city, and country attributes of two Address objects.
00:17:40.200
The other class, GPSLocation, captures latitude and longitude, with its own equality check.
00:17:46.830
Now, let’s align these two classes to extract similarities that will help us build our generic solution.
00:17:54.080
The differences are trivial, so we will focus on code that checks equality by comparing the attributes.
00:18:03.180
We can create an array of keys representing attributes in both classes and then utilize the all method to verify equality.
00:18:11.510
Next, we will create a method called 'equalize,' distinguishing keys as our unknowns across the classes.
00:18:18.410
The use of defined_method allows us to dynamic create methods based on our array of keys.
00:18:25.920
This, however, falls under the previous working definition of metaprogramming, as it converts unknowns into functionalities.
00:18:33.170
No change occurs aside from our extraction of commonality using the module called Equalizer.
00:18:40.870
We have successfully derived a gem that allows us to call equality checks for both classes through a unified, generic interface.
00:18:48.460
This method now provides a more efficient approach and makes our code reusable across different objects with minimal adjustments.
00:18:56.660
We can build on this abstraction further, creating methods for inspecting attributes, returning and checking hashes associated with attributes.
00:19:03.480
This modular approach allows for easy further development and enhancements to our applications, striking a balance of performance and readability.
00:19:09.860
Lastly, what I want to stress is that all of the concepts we've talked about fall under the broader umbrella of generic software.
00:19:17.290
Dry Equalizer is a gem built upon these principles, implicating that we have already implemented much of this work before.
00:19:23.860
In addition, taking a glance at the DryRB ecosystem demonstrates how these components harmoniously interact.
00:19:30.960
As we reach the end of this talk, let's return to the title, 'Metaprogramming for Generalists.'
00:19:37.360
The term generalist often refers to someone with broad knowledge across fields, but I defined it differently.
00:19:43.920
In my interpretation, a generalist is someone willing to dig deeply into core concepts that address various problems—like attributes and equality.
00:19:50.700
To summarize, we need more community members to become these generalists who can build and share generic software.
00:19:57.350
By solving larger, generic problems, we make specific instances easier to handle.
00:20:04.690
This is the magic of metaprogramming—the metaphor that forms the backbone of the Ruby ecosystem.
00:20:11.350
That's what makes Ruby the powerful language that it is.
00:20:19.200
Thank you!