00:00:12.280
I'll be speaking at a conference outside of Japan for the first time since COVID-19, and I am happy to present my talk at Euruko. Today's topic is how to resolve gem dependencies in your code. My name is Hiroshi Shibata; please call me Hiroshi. I primarily work as an open-source software developer at Anot. I am a member of the Ruby C team and the RubyGems and Bundler teams, working on maintaining the Ruby ecosystem and its infrastructure every day. Thank you for using Ruby, RubyGems, and Bundler.
00:01:06.280
Let me introduce my company, Anot. Anot provides software products for the construction industry, and our service is built with Ruby, Ruby on Rails, and JavaScript. Anot is a supporter of the Ruby ecosystem, and they hired me as an open-source developer. I've traveled over 18 hours from Japan to be here at EUR today. Japan is a significant place for the Ruby language, and I am excited to attend in Lithuania.
00:01:35.320
I would like to introduce RubyKaigi, one of the major conferences for the Ruby language. It is a great opportunity to meet many Ruby developers and experts. You will be able to engage in discussions about Ruby and enjoy delicious food and drinks. RubyKaigi 2024 will take place in Okayama, Japan, a beautiful area with scenic views of the sea and islands. I look forward to seeing you again at RubyKaigi in May 2024. Please submit your proposals!
00:02:16.440
Now, let me proceed to introduce the first topic of my presentation: dependency resolution with RubyGems and Bundler. First, I will provide an overview of RubyGems and Bundler. A package manager like Bundler serves two primary functions. First, it offers a user-friendly interface for users to install, upgrade, and uninstall packages. Second, it undertakes the task of resolving dependencies among package names.
00:03:40.720
Additionally, a crucial feature is version locking. This functionality ensures that a consistent list of packages is available regardless of the unique processes. RubyGems is the standard package manager for the Ruby language. It is developed within the GitHub organization, consisting of two primary repositories: one is RubyGems, which is an application for hosting Ruby libraries, and the other is the command-line tool for RubyGems.
00:04:49.320
My main focus remains on the RubyGems C repository, where I work with it daily. Next, I will introduce some terminology. A 'gem' is a unit of external library software or frameworks, and a 'gem specification' is a class that defines metadata, including the gem name, version, platform, and dependencies. A gemspec file describes the gem specification and can be created manually for gem releases or automatically at the time of gem installation.
00:06:05.880
Now, let’s take a look at Bundler. RubyGems provides a user interface and dependency resolution, but it does not have the capability of version locking. Bundler provides this version locking functionality, which is achieved using a Gemfile. You may be familiar with the Bundler application. Let's look inside the Ruby libraries and how Ruby loads gems. You can check the gemspec, which is the metadata of a gem, using the 'gem specification' method with the Gemfile.
00:07:20.919
For example, the Rack gem contains various metadata, with the most important being the dependencies section. In this example, the Rack gem has four dependencies: 'minitest', 'minitest-spec', 'rack-test', and 'rake'. These dependencies are marked as development dependencies, meaning you can use Rack without needing the other dependencies. Developer dependencies are unnecessary for your application or code, as they show the metadata that RubyGems and Bundler use to resolve all the gems and versions required.
00:08:15.360
In Ruby, there is a slightly different kind of gem called default gems. Default gems are gems that can be required regardless of the Gemfile, and they can be updated with the 'gem update' command. You can check if a gem is a default gem by referring to the gemspec. For instance, you can require the RSS library and check its specification using `gem specification`. For example, when checking RSS, it returns false indicating RSS is not a default gem, but for OpenSSL, it returns true, confirming that OpenSSL is a default gem.
00:09:17.880
Furthermore, Ruby has 'bundled gems' which are automatically installed during installation without needing to run 'gem install' or 'bundle install'. This is exactly the same as your local installation of gems such as Rails or other libraries. I work on organizing the bundled gems in each Ruby release package every year. Bundler only requires gems from the load path environment variables.
00:10:00.960
However, Bundler can automatically add the gem path even if the path does not exist in the load path variables. Bundler extends the original require method of Ruby so that it injects the required paths and libraries. As a result, the require process is slower compared to the original Ruby require methods, creating a typical slowdown. Bundler has mechanisms to address this slowdown, which will be discussed in this talk. Bundler also extends various classes and methods of RubyGems, affecting the moment Bundler is initialized. For example, the extension defines the directory of the essential library in the gem specification class.
00:11:22.799
Bundler must locate the gem repositories, which can be specified in the gemspec or Gemfile. This section introduces the architecture of RubyGems and Bundler. Both have similar functionality and structure, using resolution engines for dependency management. RubyGems uses the Morino resolution engine, while Bundler uses P-graph, which is a different implementation. There are also differences in the command line interface and runtime architecture between RubyGems and Bundler.
00:12:06.639
Resolution is the process of creating a condition based on the combination of multiple libraries. The resolution engine provides a low-combination result based on library names and versions. Finally, the gem resolver is a mechanism that abstracts the necessary data for the resolution engine, providing a list of libraries to be installed for RubyGems and Bundler.
00:12:45.600
Now let's understand how Bundler resolves gem versions and updates the load path. We will use a Gemfile that specifies a single dependency on the RSS gem, which itself has a dependency on rexml. When you run 'bundle exec' with this setup, it will call the Bundler setup method to set up the environment variables and modify certain environment configurations such as the load path variables. After verifying the Gemfile, Bundler will install any gems defined and listed in your lock file.
00:14:26.680
Bundler also performs cleanup on unnecessary paths in your load path when using Bundler. If unused paths remain, those will be rejected. Bundler utilizes all directory specifications associated with names and versions as they are imported from the Gemfile. This process allows Bundler to resolve the necessary gem dependencies. In this case, Bundler should resolve and return all dependencies for RSS, rexml, and itself as specified in the Gemfile.
00:17:30.200
In the second part of my talk, we will explore how Bundler expresses the gems needed for your application and updates the necessary versions. Let’s take look at how the resolution engine, P-graph, experienced various updates and developments. P-graph is built to resolve dependencies quickly while handling the complexity of packages. It was originally developed for a different package manager, but has been implemented in many languages, including Ruby.
00:19:15.000
P-graph is designed to skip resolving conflicts in the graph efficiently, allowing it to run faster than Morino. An example of how P-graph works is when there is a conflict amongst the defined versions of gems, it gives up easily when the dependencies cannot be found due to version incompatibility. P-graph prioritizes efficient resolution over complexity, making it easier for developers to manage their gem dependencies.
00:22:47.360
This example showcases gem dependencies defined within a Gemfile. If certain dependencies have conflicting version requirements, P-graph will provide explicit exceptions to indicate that resolution cannot occur due to incompatible dependencies. As a result, Bundler must iterate through the potential resolutions, reset the process, and attempt re-routing to resolve any dependency conflicts.
00:24:41.320
As you have seen, maintaining a clean and functional resolution system for gems can be deeply complex. Not only does Bundler involve itself in managing the dependencies, but it also updates the required gems dynamically as needed. Dependency management in Ruby is challenging, especially when applications can scale into numerous gems and dependencies. This framework encourages best practices and optimal handling of library dependencies.
00:28:24.000
In conclusion, I would like to summarize the essential points from my talk about RubyGems and Bundler. I appreciate the opportunity to present today and share insights about making Ruby a better language for developers. Thank you for your attention; I'm excited for the continuing support of the Ruby ecosystem. Remember that Ruby is built to be a programmer's best friend, and I will ensure it continues to be regarded as such. Thank you all!
