00:00:10.120
Hi everybody, thank you so much for being here today. I am Ridhwana Khan, and I'm excited to talk to you about demystifying some of the magic behind Rails.
00:00:16.680
I'm based in Cape Town, South Africa. I am a lead software engineer, a technical writer for the official Ruby on Rails guides, and a community builder. But most importantly, I'm a cat mom to these two goofballs. This is Zeus—least attractive photo I have—and this is Zoe.
00:00:27.160
Zoe sometimes helps me write the technical Ruby on Rails guides, but she's always distracted by bird videos in the background, and she makes so many typos. So if you're reviewing any of those documentation PRs and you come across typos, you know who to blame—not me!
00:00:44.520
Good documentation is crucial; it's the primary source of truth for developers. Rails is known to be well documented, but there's always room for improvement, and that's where the technical writing team comes in.
00:01:03.960
The Rails Foundation has assembled a team to thoroughly review the documentation, filling in gaps, making sure that the content is up to date, and writing clear and accessible documentation for developers at all levels.
00:01:17.759
It's a three-step process: we audit the entire Rails documentation and the open issues that are labeled 'docs' to identify whether there's missing or outdated content. The sections are assigned to writers, and once the internal PRs are opened, at least three other team members review that PR.
00:01:30.079
After reviewing it, we polish it, and thereafter we create a PR on the Rails repo. It's marked 'RF docs' and has a label 'Rails Foundation', and that's where you come in.
00:01:41.280
It's open for a week for community review to write accurate and helpful documentation. I often dive into the real source code to understand how the underlying components work. Hence, I'll start this presentation by using my experiences to share some of the strategies and patterns that I found useful when navigating the Rails source code.
00:01:58.840
Next, we'll dive into three examples in the Rails codebase that are used to dynamically generate methods. We'll discuss metaprogramming techniques like `method_missing`, `class_eval`, `instance_eval`, `send`, and others, and we'll explain how these methods work together to create powerful and dynamic behavior in Rails.
00:02:15.600
We'll also review the benefits and drawbacks of using metaprogramming in Rails, highlighting how these techniques impact the framework. Finally, I'll briefly talk through use cases and how we've applied the patterns in a recent project.
00:02:28.000
By the end, you'll have tools to trace through the code, recognize patterns, and hopefully demystify some of the magic behind Rails.
00:02:40.360
The Rails codebase can be really overwhelming at first; however, with the right approach, it becomes more manageable. In this section, we'll explore strategies to make navigating the source code a little less intimidating.
00:02:54.280
When I'm diving into a new area of the codebase, I like to approach it systematically. I start first by consulting the Rails guides, which are invaluable for understanding what a component does and how it is used.
00:03:14.560
For example, if you're curious about the `belongs_to` association, the Rails guides provide a detailed explanation of its usage and behavior. The API docs, however, point you to specific methods and classes involved, as well as the source location of these methods within the Rails codebase.
00:03:28.480
It's important to understand a method's intended behavior before trying to decipher its implementation for broader understanding. I also read the READMEs of each component; these files offer an overview of the purpose as well as how components integrate within the framework.
00:03:44.680
For example, the README for ActiveModel provides insights into how the library facilitates object models without needing a database.
00:03:58.640
I mentioned that you can find the source location in the API; however, you can also leverage Ruby's `source_location` method to find where that method is defined.
00:04:16.720
For example, if you're trying to figure out where the `find_by` method is implemented in the Rails console, you can run something like `user.method`. You can pass it the method you're trying to find—in this case, it's `find_by`—and then you call `source_location` on it.
00:04:31.840
It will give you back the file path as well as the line number, which is really useful for tracing through the code. You can open that file and further examine how Rails handles this dynamic query generation.
00:04:44.680
It's important to pay attention to recurring patterns and conventions in the Rails codebase. There's a metaprogramming technique called `define_method`, which is used to create methods based on certain conditions.
00:05:01.680
If you search for `define_method` in the Rails codebase, it appears 260 times across 110 files. This frequency highlights how foundational it is to the Rails codebase, making it crucial for you to understand in order to navigate Rails source code effectively.
00:05:13.760
Recognizing these patterns can help you understand similar occurrences in other parts of the codebase. You can also use the command `bundle open` to view a specific component within your editor.
00:05:26.000
This, combined with something like the debug gem, will allow you to set breakpoints to trace through the code and inspect variables using a debugger.
00:05:38.080
Rails is also known for its robust testing suite. Reviewing tests can reveal expected behavior as well as help you understand the use cases and edge cases.
00:05:54.640
For instance, one of the challenges when working with metaprogramming, like `define_method`, is ensuring that you correctly understand what's happening because metaprogramming can mean that method definitions are often abstracted away.
00:06:06.320
Here's an example of a spec that confirms the dynamic generation of validation methods, which is implemented using `define_method` in the source code. These tests ensure that the appropriate validation methods are generated based on different types of associations.
00:06:19.600
By reading through these tests, you can better understand the underlying code. Take advantage of AI tools like GitHub Copilot to assist in code exploration.
00:06:32.480
For instance, if you're working in a model file and you're unsure about how the validation in that method works, Copilot may be able to explain the relevant code to you and point you in the right direction.
00:06:48.840
However, that doesn't mean you shouldn't critically review the suggestions that Copilot gives you and ensure they align with your understanding and your own research.
00:07:04.960
By combining these techniques, however, Rails will become less of a mystery, making it easier to debug, develop, and deepen your understanding of the framework.
00:07:20.080
To understand the Rails source code, one should aim to be familiar with Ruby metaprogramming because Rails uses so much of it. Metaprogramming allows a program to write or modify its own code while operating at runtime.
00:07:34.080
That is, when a program is actively running, metaprogramming makes the Rails source code more flexible and adaptable to different situations without needing to change the underlying codebase.
00:07:46.720
There are numerous metaprogramming techniques and patterns embedded within the Rails framework. In this section, I'll guide you through examples in the Rails source code that use these patterns, illustrating their role in the Rails framework we know and love today.
00:08:01.920
Example one is an excellent illustration of how `method_missing` and other metaprogramming techniques are utilized in Rails. This module is critical for handling MIME types in controllers and it manages the collection and organization of different response formats for a given controller action.
00:08:15.040
When you use `respond_to` in a controller, it matches the requested MIME type and then executes the corresponding block of code. Consider the Articles controller: in this controller, we have a show action that retrieves a single article depending on the `GET` accept header of this request.
00:08:30.720
The top right of the slide indicates the content type the client is able to understand and would respond with the relevant MIME type. In all cases, the collector module is responsible for determining the appropriate response format.
00:08:44.960
Now, this is a lot of code on the screen, but we're going to go through the important bits—the ones that are dynamic in nature. Just slightly intimidating looking and recurring patterns across the entire codebase.
00:09:01.440
These are the two important bits: `generate_method_for_mime` and `method_missing`. An example we will use to trace through the code is a `GET` request to an article show method requesting JSON.
00:09:18.000
You can find it at the top right corner of every slide. When a client requests an article in JSON format, the response block in the controller tries to find a method named `json`.
00:09:32.320
Since `json` is not explicitly defined, `method_missing` in the collector module is triggered. So, what is `method_missing`? `method_missing` handles calls to methods that are not explicitly defined in the codebase, thus enabling dynamic method creation.
00:09:45.680
When `method_missing` is executed, it is passed a parameter symbol, which, in this case, is the MIME type format JSON. Since JSON is a registered MIME type, it eventually calls a method named `generate_method_for_mime`.
00:10:02.240
Using our previous example, `generate_method_for_mime` dynamically generates instance methods for the given MIME type. It uses `class_eval` to dynamically define a method after the symbolized MIME type.
00:10:16.880
`class_eval` allows you to evaluate a block of code within the context of a specific class or module, enabling you to modify or extend a class's functionality without altering the class's original source code.
00:10:30.320
So, this dynamically created method `def find_by` class eval is named after the MIME type symbol JSON. In our case, the method takes a variable number of arguments and inside it calls another method, passing the MIME type lookup argument JSON.
00:10:44.480
Once this method is defined, we trigger `method_missing` and then trigger `public_send`. `public_send` is used to invoke public methods dynamically by their name at runtime.
00:11:01.440
When `public_send` is called with the parameter JSON, we are basically calling the method JSON, which contains a custom method we pass through with the MIME type JSON.
00:11:15.520
I encourage you to trace through the custom method on your own; it processes the request based on the MIME type and the given parameters, resulting in a response hash.
00:11:30.040
Hence, when a request is made to the show action of the Articles controller, Rails checks the accept header of the request to determine the desired response format.
00:11:42.880
If the response format is JSON, Rails will look up the MIME JSON in the response hash it created, find the proc object, and execute it, rendering the article as JSON.
00:11:58.800
By combining `method_missing`, `class_eval`, and `public_send`, Rails can dynamically define and invoke methods based on the requested MIME type, making the controller's response handling flexible and efficient.
00:12:15.920
In our next example, we'll talk through building associations. Recently, when I was writing the Active Record associations documentation, I updated a section about the API methods for an association.
00:12:32.240
For instance, when you declare a `belongs_to` association for any class, Rails provides the declaring class with numerous methods related to that association.
00:12:46.560
For example, when you're defining a Post class that has a relationship `belongs_to` Author, you can use those generated methods for dynamic querying and retrieval of the association.
00:13:01.160
So these are some of the methods you'll see: you can get an author using `post.author`, and you can set it using `post.author = author`. There are numerous other generated methods like `build_author`, `create_author`, `reload`, `reset`, and track changes on associations.
00:13:14.560
The author method is also dynamically generated. Since we know that these methods cannot be hardcoded, it led me to explore how Rails generates these methods for a class.
00:13:30.120
The relevant methods are defined in this module, ActiveRecord::AssociationBuilder, and the class is defined in the Association. We're going to go through the `build` method.
00:13:44.280
The way I found this code was by using the trick I showed earlier: the `source_location`. That led me to this particular method. It's evident that the Rails source code puts a lot of effort into naming things very clearly.
00:13:58.520
So just by looking at this code and reading through it, you can see that we check for a name conflict, then we create a reflection, then we define methods like the accessors, the callbacks, the validations, and the tracking methods, and finally we return a reflection.
00:14:12.920
I've mentioned 'reflection' a lot, so what is a reflection? A reflection in Rails represents metadata about an association between two Active Record models.
00:14:26.880
It contains details such as the association class, the type, and other options like foreign key constraints. For example, when you define an association like this: a post belongs to an author and has many comments, Rails creates reflection objects for the associations 'belongs_to' and 'has_many'.
00:14:43.360
This allows Rails to understand how a Post model relates to an Author model and how it relates to the Comment model, thus enabling features like eager loading and validations.
00:14:59.200
The Active Record Reflection module provides methods to access these reflection objects. For instance, you can retrieve the reflection in an association using the method `reflect_on_association` and passing your association.
00:15:15.760
Now we're able to see what type of macro it is, what the class name of the reflection is, etc. Going back to our original code in the Association class, we create the reflection.
00:15:31.760
Now that we know what a reflection is, we understand that it encapsulates metadata about the association. It is a fruitful side quest to go through the `create_reflection` method on your own, exploring the Active Record Reflection module.
00:15:46.760
But for the purposes of this section, I'm going to focus on the `define_accessors` definition. The `define_accessors` method is responsible for creating getter and setter methods for associations.
00:16:03.440
So, remember when we talked about `post.author`? Setting `post.author = author` involves these dynamically created methods. The method defines a mixin variable and uses the generated association methods.
00:16:17.840
If we had to peek into the Active Record core module, we'd see that this method returns a dynamic module. In the case of our Post example, this module would store instance methods for associations.
00:16:35.760
In her talk, 'The Magic of Rails' at Rails World 2023, Eileen Yel described this behavior and highlighted how the magic of Active Record and associations comes from methods, classes, and modules that are dynamically generated when your application boots and when your models are loaded.
00:16:50.600
Once this dynamic module in this code is returned, we define the readers and the writers and pass through that dynamic module. We also pass through the association name.
00:17:07.040
So this is what the defined readers and defined writers methods look like. Let's discuss the reader method first. This method is used to dynamically create reader methods for associations in this dynamic module.
00:17:22.040
For a post module, `define_readers` will define a method with the association name `author`, and then it looks up the `author` association in the association cache and calls `reader` on it.
00:17:39.440
With the defined writers, we do the same thing. In our example, it will define a method `author =` and in the Post class, it will then write the `author` association for the association object.
00:17:56.000
This dynamic method definition approach mirrors the pattern we've seen in the previous example with the MIME types. In this example, however, it facilitates adding these methods to the dynamic module, `post_generate_association_methods`, and to create these association objects.
00:18:13.440
We also use this approach to define additional generated methods that I mentioned at the start, such as `author.changed?`, `author.previous_changed?`, etc. You can see it uses a similar pattern.
00:18:29.360
Additionally, for methods like `reload_author`, `reset_author`, and others, they all use `class_eval` to define dynamic methods. Once you understand how one part of the codebase works, it's easier to apply those techniques to other parts.
00:18:45.840
Our final example in this section is the `define_autosave_validation_callbacks`. This module manages automatic validation and saving of associated records when their parent is saved.
00:19:02.480
Consider, once again, a Post model that belongs to an Author. When you save a post instance, Rails will eventually call a method named `define_autosave_validation_callbacks`.
00:19:17.440
This method dynamically creates a method that will validate the associated records for an Author when saving a Post.
00:19:32.240
This process is crucial during the validation phase of the Post model to ensure that all the Author records are valid, ensuring data consistency and preventing invalid associations from being saved in the database.
00:19:49.040
Let's explore the important bits of `define_autosave_validation_callbacks`. The parameters that are passed through are a reflection, which we know now is just metadata about an association.
00:20:04.080
In this case, it's a `belongs_to` reflection based on the fact that a Post belongs to an Author. This method does a few things, which I encourage you to trace through on your own.
00:20:18.040
The part I'm going to focus on is where the magic lies: the `define_non_cyclic_method`. When we dive into this method, we realize the validation method parameter represents the name of the method that will be defined.
00:20:32.080
The validation name is defined a few lines up as `validate_associated_records_for_author`, and the block passed into `define_non_cyclic_method` here is the `send` method.
00:20:47.760
The `send` method is a metaprogramming technique that allows you to call private or public methods dynamically by name. Let's go through the `define_non_cyclic_method` to see how the validation method works.
00:21:02.640
If a method with the name `validate_associated_records_for_author` is not already defined, then we dynamically define it. `define_method` offers a way to create methods at runtime based on specific conditions or inputs.
00:21:16.640
This method definition is key to allowing Active Record to handle various types of associations without requiring countless predefined methods.
00:21:30.080
The newly defined dynamic method `validate_associated_records_for_author` also prevents cyclic dependencies by ensuring that each validation process only runs once per method call.
00:21:45.680
Once a validation method is called, it will not be called again, thus preventing infinite loops. A crucial part of how this is done is evidenced by a line of code: `result = instance_eval`, passing through the block that's assigned only if `validate_associated_records_for_author` has not been called before.
00:22:00.480
In Ruby, `instance_eval` allows you to evaluate a block of code or a string within the context of a specific object. This grants access to that object's private methods and instance variables.
00:22:16.320
This allows you to dynamically extend or manipulate an object's behavior without altering the public interface directly. Thus, the result is obtained by evaluating the block within the context of the instance.
00:22:32.880
Now, the block of code we're referencing is the `send` method, which calls the `validate_belongs_to_association` while passing in the reflection.
00:22:49.440
Using `instance_eval`, we ensure that `validate_belongs_to_association` has access to its private methods and instance variables, thereby allowing it to operate as if it were defined directly on that instance.
00:23:05.280
Returning to our original method in `define_autosave_validation_callbacks`, this line of code helps Rails dynamically register the validation method we've just discussed, `validate_associated_records_for_author`, which in turn will call `validate_belongs_to_association`.
00:23:22.960
The latter will validate all the association records. This process and the patterns used ensure Rails properly registers the dynamic validation method and performs necessary checks to validate the associated record before saving the Post.
00:23:37.840
Thus, we ensure data integrity in the application. Now that we've gone through some examples, let's talk about some of the benefits and drawbacks of using metaprogramming.
00:23:53.520
Metaprogramming allows Rails to automatically generate code and behavior, creating conventions for methods, associations, and validations dynamically, which reduces the need for boilerplate code and configuration.
00:24:10.720
That's what we love about Rails! However, Rails applications often need to handle a variety of use cases, making our lives as users of the framework easier.
00:24:26.400
Metaprogramming techniques provide the flexibility to define and modify methods, classes, and behaviors, making the framework extensible and flexible.
00:24:42.480
It also helps to simplify the design of the Rails API by allowing the framework to provide a more intuitive and expressive interface, without exposing the complex underlying implementations.
00:24:58.960
Finally, Rails leverages metaprogramming to adhere to the Don't Repeat Yourself principle by generating repetitive code patterns dynamically. This keeps the codebase clean and reduces redundancy.
00:25:15.040
However, despite these benefits, there are drawbacks. Metaprogramming can add layers of abstraction, making code less transparent, which can lead to difficulties in debugging features and understanding how different parts of the code interact.
00:25:31.440
The dynamic nature of metaprogramming can also introduce performance overheads. Generating code on the fly and evaluating code at runtime may be slower than static code, potentially affecting the performance of applications.
00:25:46.720
Lastly, code that heavily relies on metaprogramming might become harder to maintain, and it may be difficult to attract new contributors, as they may find dynamic code intimidating.
00:26:05.760
However, while there are challenges, Rails leverages metaprogramming in a way that enforces strong conventions, minimizes configuration, and introduces adaptability, which ultimately improves the developer experience.
00:26:22.720
Learning and applying metaprogramming from the Rails source code can greatly improve your ability to build dynamic and flexible systems.
00:26:39.120
There are numerous situations where it's beneficial. For example, when I worked at a company called Forum, also known as Dev.to, which created community software, we used metaprogramming to create dynamic profile fields for different communities.
00:26:55.160
If there was a gardening community, we might need to ask users, 'What is your favorite plant?' and that would be a profile field, whereas for a developer community, you might want to ask users if they're available for hire.
00:27:12.440
Each of these dynamic fields could have different data types like strings, booleans, etc. This is the code that resides in that open-source repo—the Forum open-source repo.
00:27:27.600
I won't go through it, but you can read through the code if you wish. It uses a combination of `method_missing`, `send`, and `class_eval` to create dynamic profile fields.
00:27:45.760
I hope that by exploring the Rails source code, it has inspired you to dive deeper into Rails. We've gone through examples and seen how metaprogramming enhances Rails flexibility.
00:28:05.280
As you continue to work with Rails, consider how these concepts can also be applied to your projects. Understanding the internal mechanics of Rails will help resolve complex issues, as well as open up possibilities for enhancing your applications.
00:28:21.920
If you have the time, I would strongly encourage you to engage with the Rails community. Review pull requests, whether they are active or whether you're reviewing them passively, and contribute to the community.
00:28:36.320
Not only will it deepen your understanding of Rails, but it will also strengthen your connection with the Rails ecosystem.
00:28:53.760
As I wrap up this talk, I'd like to thank Patrick Deos for reviewing the talk multiple times, as well as the Rails World team for this opportunity.
00:29:08.080
But most of all, I'd like to thank all of you for coming today to join me. I look forward to chatting with some of you as the day progresses, and you can find me on X with the username @RidhwanaK. Thank you so much!
