Talks

Zeitwerk Internals

Zeitwerk Internals

by Xavier Noria

In the video titled "Zeitwerk Internals," Xavier Noria, a core member of the Rails team and the author of the Zeitwerk gem, presents an in-depth exploration of how Zeitwerk functions internally at the Friendly.rb 2023 conference. This presentation focuses on the importance of Zeitwerk as an autoloader, reloader, and eager loader for Ruby code, emphasizing its extensive usage and transparency within Ruby projects.

Key Points Discussed:
- Introduction to Zeitwerk: Zeitwerk was published in 2020 and has achieved significant popularity, with approximately 300 million downloads and around 50 releases. It supports both Rails and Hanami, enhancing various Ruby applications.
- Functionality and Independence: Zeitwerk operates independently, enabling multiple loaders to coexist within a Ruby process. This design helps manage separate source trees without user awareness.
- Transparency in Usage: The gem is built for transparency, ensuring users do not need to document its use. It interacts seamlessly with the user's code and requires minimal configuration.
- Understanding Ruby Constants: Noria explains Ruby's unique handling of classes and modules, emphasizing that these are treated as constants, which forms the backbone of the autoload mechanism.
- The Autoload Mechanism: The autoload method in Ruby allows for deferred loading of constants, which Zeitwerk utilizes to enhance application performance by only loading files as needed.
- Integrating Zeitwerk with Applications: Zeitwerk allows easy setup where developers specify root directories for managing namespaces, which is essential for both standalone Ruby gems and larger applications like Rails.
- Efficiency and Performance: The lazy loading feature of Zeitwerk improves performance by only requiring files when their constants are referenced, optimizing resource usage.
- Final Observations: Noria encourages developers to follow naming conventions and file structures to maximize the benefits of Zeitwerk in their projects, anticipating its impact across various implementations.

Conclusions: The presentation highlights Zeitwerk's capabilities as a crucial tool for Ruby development, providing a clear, effective, and transparent loading system that aids in code organization and performance. Noria hopes attendees will apply their learnings about Zeitwerk in their future projects.

00:00:08.400 Our first speaker is actually the reason why we are here. When I wrote to him back in the winter, I think in March, I didn't have a venue, a date, or any details. But I told him about the kind of event we wanted to build, and he responded with, "I love it; count on me!" That was just amazing. You'll find the same level of kindness and friendship when I talk to him on GitHub or other places about tech. He's a core Rails team member and the author of Zeitwerk. Please welcome Xavier Noria!
00:00:42.559 Thank you! It's a great pleasure to open the first edition of Friendly.rb. Organizing a conference like this is not a small task, so the organization has been extraordinary. Please give it up for the organizers!
00:01:07.720 A little bit about me: Adrian already mentioned that I'm a core team member and author of Zeitwerk. I have been honored to receive a few awards related to Ruby, which I've pursued in my spare time. For a living, I do freelancing.
00:01:21.360 Now, some remarks about Zeitwerk. It's an autoloader, reloader, and eager loader for Ruby code. It was published in 2020 and has seen about 50 releases so far, approaching 300 million downloads. These statistics are from RubyGems, which includes activities like building Docker images and running CI, indicating it's being widely used. Zeitwerk powers Rails and Hanami, and it's crucial to note that while it has a strong connection with Rails, Zeitwerk is an independent project designed to work with any Ruby project.
00:01:44.720 The last point is important because the gem was designed to allow multiple loaders to coexist in the same process, managing their independent source trees. For instance, in a Rails application, there are two loaders. While we typically recognize only the main loader, the secondary loader may also rely on Zeitwerk. You might not even realize they are using Zeitwerk, as one of the project's goals is transparency.
00:02:06.720 By transparency, I mean that in gems, you don’t have to document that you are using Zeitwerk. This is internal and doesn't require special APIs or anything unusual. You only need to say, "require my entry point," and from the outside, there's no indication that Zeitwerk is being used. Additionally, when you use the library, you only need to make a few calls to configure your application, but after that, your project is just your code, your Ruby, the way you like it. The only thing you must comply with is the naming conventions, but beyond that, Zeitwerk operates entirely in the background, designing to maintain this transparency.
00:02:35.200 Now, while transparency is fantastic for user experience, I believe it is important that this is not a black box. It is better for users to enjoy what the gem does while also understanding what happens behind the scenes. Thus, the goal of this talk is to explain how Zeitwerk works internally.
00:03:04.400 Zeitwerk is a constant autoloader, eager loader, and reloader. To explain how it works, we need to align on some technical concepts in Ruby. There will be an initial section with some remarks relevant for understanding.
00:03:18.000 Firstly, it's essential to note that the class and module keywords store class or module objects in constants. This is unique to the Ruby programming language. For instance, consider the two code snippets: the first snippet creates a class, while the second snippet demonstrates that a class object is created and stored in a constant. Likewise, the module keyword functions similarly.
00:03:39.720 Now, related to this is the understanding that Ruby does not have syntax for types, which is another unique aspect of the language. If we store zero in a constant X and check if X is even, we find that it returns true. Therefore, when you reference a class or module, you are interacting with constants, not types. For example, when you refer to 'Project', that's not a type; rather, it is a constant holding a class object.
00:04:14.000 Another significant aspect of Ruby is that constants belong to classes and modules, contributing to the emulation of namespaces. Internally, you can think of a constant table as mapping simple constant names to their values. Every constant you encounter in the code belongs to some class or module, thereby grouping them within the namespace of the containing class or module.
00:04:54.720 So, suppose we define a constant C in a class or module. If we check the constants of that module, we will see the constant C listed among others. It's important to recognize that anything written with capital letters in Ruby is treated as a constant, which distinguishes it from variables.
00:05:16.000 Now, to connect this to the implementation of Zeitwerk, we encounter the use of autoload. The autoload method allows you to define methods that will automatically load the constants when they are referenced. For example, in a module, if you define autoload constants, upon their first reference, Ruby will automatically require the corresponding files.
00:05:39.760 Autoloading provides a way to defer the loading of files until they are needed, thus improving application performance by not loading everything upfront. If a piece of code requires constants that haven't been loaded yet, Ruby can handle that by triggering the autoload mechanism.
00:06:08.360 When we delve into the Zeitwerk source code, we learn how the autoloading mechanism orchestrates the loading of constants. Essentially, the way you use Zeitwerk in a project is very straightforward. You instantiate a loader and specify the root directories for the namespace you want the loader to manage. By default, this namespace is object, which corresponds to the default top-level namespace in a normal Ruby program.
00:06:41.280 After you call the setup method, everything is ready for autoloading, eager loading, or whatever else you want to do with your code. In Rails applications, you do not need to perform this setup manually because Rails integrates this setup process. Essentially, Rails runs this setup behind the scenes as part of starting the application.
00:07:09.640 However, for the generic interface in Ruby gems, there is typically only one root directory. That's part of the functionality Zeitwerk provides, allowing a streamlined setup process for different applications.
00:07:30.920 Now, the core of Zeitwerk revolves around how autoloading works. When you reference a constant that hasn’t been loaded, Zeitwerk's autoloading mechanism kicks into action, requiring the file that defines the constant.
00:07:56.200 Imagine a small project with a users controller, a namespace for admin, roles controller, and models for user and admin role. Zeitwerk needs to define autoloads for the users controller and user model, allowing the corresponding classes to be loaded only when referenced. An important aspect is that if you add, remove, or edit the files in your project, Zeitwerk keeps track of these changes, ensuring efficient and relevant loading.
00:08:31.720 So, to set up autoloads, Zeitwerk iterates through the root directories you've specified, defining autoloads for files and subdirectories. This approach allows for efficient organization of code, reducing load times by only loading the necessary components when explicitly referenced.
00:09:00.280 Importantly, autoloading happens lazily, meaning it won't load files until their associated classes or modules are actually referenced in your application code. Thus, when you execute your application, only the files that are needed are loaded, contributing to overall performance efficiency.
00:09:28.320 In conclusion, Zeitwerk offers a powerful, transparent, and efficient way of handling code loading in Ruby applications, whether they're small scripts or large frameworks like Rails.
00:09:58.320 That said, as the conference progresses, it's important to note that while we enjoy this new functionality brought by Zeitwerk, developers must adhere to naming conventions and file structures to take full advantage of its capabilities.
00:10:25.000 I am excited to see how attendees leverage Zeitwerk in their own projects as we dive deeper into understanding its internal mechanics. Thank you for your attention, and I hope you enjoy the rest of the conference.