In the beginning, there was "require"...

by Adam McCrea

In this RubyConf 2019 talk titled "In the beginning, there was 'require'..." Adam McCrea explores the fundamental concepts surrounding the 'require' method in Ruby, illustrating its critical role in managing dependencies within Ruby applications. McCrea shares his personal journey of discovery, highlighting his initial confusion over various 'require'-like methods, particularly 'require_dependency', which led him to present this talk aimed at beginner and intermediate Ruby developers.

Key points discussed include:

- Understanding Dependencies: McCrea categorizes dependencies into three types: standard library code, Ruby gems, and internal project code. Each of these requires different management techniques.

- The Role of 'require': He explains how 'require' serves as the foundational method for loading dependencies, pointing out its functionality, return value, and the mechanism behind the load path.

- Eager vs. Lazy Loading: McCrea discusses the difference between eagerly-loaded and lazy-loaded dependencies, positioning 'require' as an eager loading method while providing a matrix to better understand these concepts.

- Requiring Gems and Local Files: The speaker describes how requiring gems (e.g., 'MiniTest') involves gem activation, where RubyGems manages the load path. He also outlines the nuances of requiring local project files, emphasizing proper load paths.

- Alternatives: Methods like 'requirerelative' are introduced for managing file requirements without relying on the current working directory, eliminating potential pitfalls.

- The 'load' Method: McCrea contrasts 'load' with 'require', explaining that 'load' requires the file extension and reloads files each time it's called.

- Bundler: The significance of Bundler is highlighted, particularly its role in managing gem versions through the 'Gemfile' and 'Gemfile.lock', ensuring consistency across different environments.

- **'requiredependency' in Rails**: He addresses how 'require_dependency' enforces eager loading of dependencies within Rails applications to prevent conflicts with auto-loading.

In conclusion, McCrea encourages developers to understand the mechanics behind these methods to fill knowledge gaps that many may encounter, regardless of experience. He underscores the importance of managing dependencies efficiently to avoid common pitfalls and enhance coding practices in Ruby programming. McCrea invites attendees to reach out for discussions or questions, emphasizing continual learning in the Ruby community.

00:00:12.869 Hello, good afternoon, everybody. So, I want to tell you a little story about why I'm here giving this talk today.

00:00:19.480 It's a little bit of a confession. Earlier this summer, I was working on a project that involved adding some custom video processing for file uploads in a Rails application.

00:00:24.310 This led me to look through the source code for Active Storage, the library that Rails uses for file uploads. While I was perusing the code, I came across some lines that puzzled me.

00:00:30.220 I was familiar with 'require' but had never seen 'require_dependency' before. I didn't understand why they were there or whether they were part of Ruby or Rails. Unfortunately, my curiosity took a backseat as I was focused on my tasks.

00:00:45.220 Fortunately, those lines were not crucial to what I was doing at that moment, so it wasn't a big deal. However, it wasn't long before I needed to package the code I was writing into an engine to incorporate it into our Rails application.

00:01:02.700 Now, while Rails provides some boilerplate code for engines, ultimately, it’s up to you to decide where to place the engine, how to require it, and how to glue everything together. I found myself in unfamiliar territory again, which made me uncomfortable due to my knowledge gap.

00:01:08.590 Feeling the pressure, I resorted to copying code—what I like to call 'cargo culting'—by relying on Stack Overflow and other resources instead of diving deep into understanding how everything worked.

00:01:19.920 I’ve been guilty of cargo culting more often than I’m comfortable admitting. Ultimately, I got the job done, but I felt ashamed and embarrassed that I hadn't taken the time to actually understand the mechanics behind the 'require' functionality in Ruby.

00:01:29.340 Despite having written Ruby applications professionally for 12 years, I lacked a foundational understanding of how to require code in a Ruby application. My goal today is to help fill that knowledge gap, which may also resonate with some of you.

00:01:40.070 Before I dive in, let me introduce myself. My name is Adam McCrea, and I go by 'AdamLogic' online, including on Twitter and GitHub.

00:01:46.200 I work for a company called You Need A Budget, also known as YNAB. We create budgeting software to assist people in gaining control of their finances, and it's truly amazing.

00:01:56.610 I also developed a Heroku add-on called Rails Auto Scale, which can simplify your experience if you’re running a Rails app on Heroku.

00:02:00.300 Now, let's talk about dependencies. In this talk, I'm referring to dependencies as any code that we need to integrate into our application.

00:02:07.149 I categorize them into three distinct flavors. First, we have standard library code, which includes items like CSV, OpenURI, and Logger. These are packaged with Ruby but must be required in order to use them.

00:02:26.060 Next, we have Ruby gems. These are pieces of code created by others who have generously open-sourced their work, providing us with valuable resources to incorporate into our projects.

00:02:35.080 Finally, we have code within our own projects. For larger applications, consolidating all code into a single file is impractical, and it's essential to break it down into modules and directories.

00:02:44.750 However, we still need to connect those files, similar to how different programming languages handle file requirements.

00:02:48.430 For instance, PHP uses 'include', Node.js utilizes 'require', and ES6 implements 'import'. In Ruby, we have 'require', 'require_relative', 'require_dependency', 'load', and 'autoload'. In Rails, everything seems to happen magically, often without needing to require anything.

00:03:18.610 Reflecting on my own journey, I was amazed by how little I understood regarding the various techniques available to us for managing dependencies. As I gained insights into these concepts, I began to fit them into a mental model.

00:03:38.000 I envisioned a matrix, with one axis dedicated to eagerly-loaded dependencies and the other to lazy-loaded dependencies. Eagerly-loaded dependencies are those for which we tell Ruby explicitly that we need them right away.

00:04:06.600 In contrast, lazy-loaded dependencies mean that we inform Ruby of our need for the dependency but postpone its loading until later. The other two axes categorize dependencies as explicit or implicit, defining whether we tell Ruby exactly where to find the files it requires.

00:04:52.470 Let's now explore the star of the show: 'require'. 'Require' serves as the foundational method for all these functionalities, and to comprehend it better, let's look at some examples.

00:05:06.320 The simplest use of 'require' is to include a standard library, such as CSV. Although CSV comes bundled with Ruby, if we attempt to reference CSV without first requiring it, we'll encounter an error. Thus, we must always use 'require' prior to utilizing CSV.

00:05:30.420 'Require' returns true, indicating that the requested file was successfully found and loaded into our application. If we try to require CSV again, it will return false, indicating that the file has already been loaded, as it's not necessary to load it again.

00:05:50.920 It's worth mentioning that calling 'require' multiple times for the same file effectively acts as a no-operation. 'Require' itself is a method that sits within the kernel module, along with many other commonly-used methods like 'puts' and 'raise'.

00:06:05.490 Because these methods are part of the kernel, they can be invoked anywhere in the program, and since they're simply methods, they can also be overridden if necessary.

00:06:28.070 When you pass a string like 'CSV' to 'require', Ruby examines a global variable known as 'load_path'. 'Load_path' is an array of filesystem paths, and 'require' will iterate through each of those paths in order to locate the file that corresponds to the name provided.

00:06:50.720 Specifically, it's looking for a file named 'CSV.rb' among other possible extensions within designated directories. In the case of standard libraries, it's quite likely to find them since they are typically included in the load path out-of-the-box.

00:07:16.240 Moreover, when Ruby successfully identifies and loads this 'CSV' file, it also adds it to another global variable called 'loaded_features'. 'Loaded_features' maintains a list of every file that has been successfully loaded by the 'require' method.

00:07:36.210 This permits 'require' to recognize that if we attempt to load 'CSV' again, it will know it doesn’t need to reload it, thus enhancing efficiency. After we require 'CSV', this 'CSV.rb' will be recorded in 'loaded_features' along with related dependencies, which might be required by the 'CSV' library itself.

00:08:01.990 Requiring a gem, such as 'MiniTest', operates very similarly. If 'MiniTest' is installed on your system and you attempt to use it without requiring it first, you'll receive an error. Once you require it, the functionality becomes available.

00:08:23.440 The procedure appears identical to requiring a standard library, but there’s a critical distinction to recognize. Before invoking 'require', 'MiniTest' is not part of your load path.

00:08:47.690 When you require the 'MiniTest' gem, it performs what is known as 'gem activation'. The 'RubyGems' system overrides the 'require' method, incorporating additional functionality for gem management.

00:09:01.210 The first action taken by the 'require' method from 'RubyGems' is to check 'loaded_features', similar to the standard 'require'. Since 'MiniTest' has not yet been loaded, it will then verify the 'load_path'. As 'MiniTest' is not on the load path at that moment, it checks for an installed gem that matches the required gem and adds its library folder to the load path.

00:09:46.620 Consequently, after adding the 'lib' folder corresponding to the 'MiniTest' gem to the load path, it resumes the traditional 'require' workflow. Now, when 'require' seeks the 'MiniTest.rb' file, it can successfully locate and load it from its designated library folder.

00:10:24.000 Moving on to requiring files from within our own projects, let's consider a simple structure with a 'main.rb' file and a corresponding 'example.rb' file situated within a 'lib' directory. If we aim to require 'example' from 'main', we may be confronted with issues.

00:10:49.750 If we attempt to simply require 'example' or 'lib/example', it is very likely to result in failure because the 'require' method only searches the load path for those files, which leads us to take alternative steps to resolve this situation.

00:11:27.740 One approach is to provide an absolute filesystem path that accurately points to the location of the desired file on our machine. While this technically works, it can be a fragile solution; should any changes occur to your filesystem structure, it becomes unreliable and poses a challenge for sharing with others.

00:12:01.080 To improve this, we can use a relative filesystem path. Although it is no longer bound to a specific machine, it introduces a potential issue; the path is relative to the working directory from which the program was executed.

00:12:38.690 If we change directories or navigate to a parent directory instead of executing 'main.rb' directly, the relative requirement may fail because the working directory changes, ultimately causing the require to fail.

00:13:00.690 So, how can we reliably require the example file in this context? A good practice could be to add the 'lib' directory to our load path. This helps when requiring multiple files, allowing for a more seamless integration.

00:13:29.120 The code to achieve that would involve adding the current file's path—specifically 'main.rb'—to the load path. By doing so, we can easily require 'example' or anything else within the 'lib' directory seamlessly.

00:14:04.050 Alternatively, we could modify our load path at runtime by executing Ruby with the '-I' option. This method functions similarly to what's described above, allowing us to require files without adjusting the load path manually.

00:14:35.240 The flexibility of adding relevant directories at runtime is particularly useful in testing scenarios, where we might want to include directories related to tests without affecting our regular load path during application runtime.

00:14:57.950 But what if we want to avoid adjustments to our load path entirely? That's where 'require_relative' comes in, offering a solution that hinges on where it is called. It’s always relative to the file that invokes it, as opposed to being dependent on the current working directory.

00:15:34.420 For example, if it is invoked from 'main.rb', it will maintain its relative context, independent of the execution directory, ensuring reliability across different runtime environments.

00:15:57.100 Moving on to 'load'. The syntax for 'load' may seem similar to 'require', but it has one major caveat: you must specify the file extension when loading. Beyond this, the primary distinction lies in the fact that 'load' reads the specified file every time it is invoked.

00:16:21.090 This characteristic renders it troublesome in cases where you aim to define constants, as it will lead to issues caused by redefining those constants on repeated calls. Consequently, 'load' is predominantly useful in temporary situations, such as IRB or creating custom loading libraries for specific needs.

00:16:39.530 Finally, let’s discuss Bundler. To comprehend Bundler's significance, we’ll revisit our previous example with 'MiniTest'. Although requiring 'MiniTest' may work well in isolation, complications arise when multiple versions of a gem exist on a system, creating ambiguity over which version is truly being utilized.

00:17:28.680 RubyGems will always default to requiring the most recently installed version of any gem. For simple scripts, this might suffice, but in shared environments, deploying code becomes complex if there are discrepancies in Ruby gem versions.

00:17:49.620 The solution Bundler provides is the 'Gemfile'. In this file, you can outline your application's gem requirements, specifying any version restrictions deemed necessary. Upon running 'bundle install', it ensures that proper versions are installed according to those specifications.

00:18:31.410 Bundler will generate a 'Gemfile.lock' file, locking your project to these exact gem versions. Any collaborators or deployment environments must adhere to this, ensuring consistency across various setups.

00:18:52.870 When you subsequently require 'MiniTest', the gem is activated and loaded from the correct version based on the defined Bundler constraints. It's essential to understand, though, that invoking 'require' prior to invoking 'Bundler.setup' could lead you to inadvertently obtain the most recent version installed rather than the one specified in your lock file.

00:19:23.490 To guarantee that you're using the desired version, ensure you call 'Bundler.require' or 'Bundler.setup' prior to requiring your gems. Bundler coordinates these load paths and gem activations effectively, offering you a robust structure for managing dependencies.

00:19:51.720 'Bundler.require' streamlines the setup by simultaneously establishing the load paths and requiring all gems defined in the Gemfile. This simplifies the dependency management process significantly.

00:20:01.870 Now, let’s take a moment to revisit 'require_dependency' before we dive into its particulars. This mechanism is particularly pertinent within Rails, and its primary purpose is to enforce the eager loading of dependencies that would otherwise be lazy-loaded.

00:20:36.040 When we enumerate three dependencies for the purpose of including them, typically, Rails’ auto-loading system handles fetching these files automatically. However, if your application requires specific modules before execution, this presents a potential risk of ambiguity. This is where 'require_dependency' comes into play, ensuring clarity and control over which dependencies load.

00:21:09.830 'Require_dependency' guarantees that the desired module is loaded correctly, preventing conflicts from the auto-loading mechanism, which might misinterpret which module to use based on the underlying structure of your application.

00:21:39.800 Interestingly, in Rails 6, the auto-loading system has been improved to automatically account for these cases, potentially rendering 'require_dependency' unnecessary in many scenarios. This advancement emphasizes the evolving nature of dependency management in Ruby on Rails.

00:22:03.010 In all, we’ve explored how 'require' assists in loading standard libraries, gems, and project files, emphasizing its importance regardless of whether you are in a Rails environment or not. Even within Rails, the basic principle of requiring standard libraries endured.

00:22:30.900 For gems written for personal use, 'require' is adequate; however, for any project intended for deployment or collaboration, we strongly recommend integrating Bundler. Utilizing 'Bundler.require' can also be a solid approach as it simplifies the requirement process.

00:23:06.390 For projects outside the Rails context, 'require_relative' is a practical alternative for ensuring more consistent module access. Furthermore, in Rails, understand the implicit behavior of the auto-loading mechanism; don't micromanage dependency loading unnecessarily.

00:23:26.590 Embrace the convenience of Rails' implicit auto-loading strategy for your application code while ensuring to require anything essential from the 'lib' directory correctly.

00:23:46.550 To conclude our discussion today, I hope I've successfully filled some knowledge gaps for you all—these gaps are common among us developers, regardless of experience level. Don’t shy away from diving into these topics, as they are vital aspects of programming in Ruby and beyond.

00:24:49.810 Knowledge gaps can often feel intimidating, but remember that they are part of the learning journey. Please feel free to reach out to me if you have any questions or wish to discuss further. You can find me on Twitter as 'adamlogic' or feel free to talk to me after the talk.

00:25:13.490 I also have some YNAB shirts and stickers if you are interested. Thank you all very much for your time today!