RubyGems on ruby.wasm

Talks

Yuta Saito

#developer-experience-dx

RubyGems on ruby.wasm

by Yuta Saito

In the talk titled "RubyGems on ruby.wasm" presented by Yuta Saito at RubyKaigi 2024, the focus is on the advancements made in RubyWasm, a project that enables Ruby to run in WebAssembly environments. Saito introduces his background, including his role as a master's student and committer for the C and Swift projects. He highlights the importance of WebAssembly in the Ruby community, noting the increase in related talks at the event and the need for a Ruby ecosystem that embraces WebAssembly.

The presentation covers the main components of RubyWasm, which include a Ruby interpreter compiled to WebAssembly, a packaging system, and an interlibrary for JavaScript. Saito demonstrates how Ruby can run in a web browser and interact with JavaScript, showcasing its versatility across various platforms including Docker and AWS Lambda.

Saito then delves into RubyGems, explaining its critical role in the Ruby ecosystem, particularly its gem installer and runtime loader, the latter of which is now supported in RubyWasm. Key points discussed regarding RubyGems include:

On-Demand Installation: This method installs gems at runtime, requiring network access and is similar to Python's micropip, though it lacks first-class support due to network concerns.
Preinstallation Support: Preinstallation of gems allows deployment without runtime network access, enabling versions of pure Ruby and C extension gems to be bundled ahead of time, enhancing performance and deployment efficiency.

The highlight of the talk is a demo where Saito showcases a Rails application (Mastodon) running in the browser, emphasizing the operational capability of Ruby in this new context. He underscores the technical challenges faced such as packaging gems, cross-compiling C extensions, and the need for linking shared libraries.

Additional technical advancements discussed include:
- The introduction of a new command 'rwm build' for packaging gem sources.
- Enhancements to 'mkmf' for better cross-compilation support.
- Development of a dynamic linking mechanism via the WebAssembly Component Model, which will streamline the build process and improve overall system performance.

In conclusion, Saito emphasizes the ongoing evolution of Ruby alongside WebAssembly technologies and extends an invitation to the community to collaborate on making Ruby a more WebAssembly-friendly language. He thanks the Ruby Association for their support, highlighting a future filled with opportunities for RubyGems within the RubyWasm project.

00:00:11.519 Okay, so today I'm going to talk about RubyGems and RubyWasm. This year, we have made significant progress in RubyWasm, and I'm excited to share the latest status with you.

00:00:30.840 By the way, this year we have four WebAssembly talks at this event. I'm happy to see the expansion of the WebAssembly community within the Ruby community. If you are interested in WebAssembly, please check out the other talks as well. Also, let's work together to make Ruby more WebAssembly-friendly.

00:01:01.079 Let me briefly introduce myself. I'm Yuta Saito, a master's student at W University, and I'm a new committer for the C and Swift projects, especially working on WebAssembly support for our package management tool, Bundler. I'm also the creator of RubyWasm. Today, I will start with a brief introduction to RubyWasm itself and the challenges of RubyGems support on RubyWasm. After that, I will show you a demo. Finally, I will share the future vision and roadmap.

00:01:52.399 First of all, RubyWasm is a project that enables running Ruby on every WebAssembly environment. This is not limited to web browsers, but it also runs on edge computing platforms, embedded devices, and as a print system. The project consists of three main components: a Ruby interpreter compiled to WebAssembly, a packaging system for Ruby files, and an interlibrary for JavaScript. Today, I will focus on the packaging system that packages the interpreter and Ruby files.

00:02:53.400 Here is an example of running Ruby in a web browser using RubyWasm. You can run Ruby code in a script tag within an HTML file, and you can access the DOM API. You can also await JavaScript promises using the await method. This allows you to write asynchronous code in a synchronous way in Ruby.

00:03:26.000 Additionally, Ruby can run on other runtimes, thanks to its portability. You can run Ruby on various platforms outside of the web browser, such as Docker, AWS Lambda, and Shopify functions.

00:03:53.159 So we can run Ruby code on WebAssembly, interact with JavaScript, and run on various platforms. But what about RubyGems? How can we use RubyGems on RubyWasm?

00:04:09.239 As you know, RubyGems is one of the critical components in the Ruby ecosystem. RubyGems has two main components: the gem installer and the runtime loader. The runtime loader enables loading installed gems at runtime. The loader itself is already supported in our version of RubyWasm, so you can use the require method with gem names as expected even on WebAssembly.

00:04:28.720 However, as of last year, only default gems and bundled gems are available due to the lack of gem installation support. RubyGems has two types of installation methods when considering WebAssembly use cases: the first is on-demand installation, which installs gems at run time, and the second is preinstallation, which installs gems in advance before running the Ruby program.

00:05:29.919 On-demand installation is convenient for daily scripting. Effectively, it runs the gem install command on WebAssembly, so it requires network access at one time. This method is very similar to Python's micropip, which is a Python package manager for WebAssembly environments.

00:05:57.880 This installation method isn’t too technically difficult, and I demonstrated it at RubyKaigi 2022. Network access can be facilitated through the browser's Fetch API, making the environment suitable for running these installations. However, we haven't added first-class support for on-demand installations yet, mainly due to network traffic concerns. Additionally, since it requires network access, it is not available on some platforms, like edge computing.

00:06:42.840 Thus, we are focusing on preinstalled support for gems. Preinstallation allows for installing gems in advance before running the Ruby program. It does not require network access at runtime, but it requires a build step before execution.

00:07:02.360 You can run gem installs on a host machine and then package the installed gems into the WebAssembly binary through a virtual file system. Currently, we can use on-demand installations for pure Ruby gems, but we don't have first-class support for C extension gems. Until last year, we didn't support preinstallation at all. In this talk, I will focus on preinstallation support for both pure Ruby and C extension gems.

00:07:53.080 Let me show you a demo for the gems approach, showcasing the quality of our gem support.

00:08:43.039 Here, you can see that we have static files with no dynamic processing involved. These are served by an HTTP server, and the system works fairly seamlessly.

00:09:51.880 The application we're running is Mastodon, a large-scale Rails application that indeed runs inside a browser. You may think it's crazy, but it's real! You can log in on the localhost and interactively create posts, demonstrating the operational capability of the Ruby environment running in this context.

00:11:04.040 While I initially considered showcasing a simple Rails app, as some others had done recently, I opted instead to demonstrate Mastodon to provide you with a fresh perspective. This is not practical for everyday use, but it serves to illustrate the potential of our gem support.

00:11:41.199 The architecture of the demo shows that the Mastodon app relies on numerous gems, including some that utilize C extensions. This is a vital demonstration of our gem support. The Rails app runs on a service worker and can respond to network requests from the web page, allowing it to communicate with an active record.

00:12:14.199 Even though this demonstration is extreme, it brings attention to what's achievable when hosting Rails apps as static sites. Such an approach could reduce costs and minimize security risks, making it an attractive option for various deployment environments.

00:12:53.360 Let me explain the technical details of RubyGem support in RubyWasm. Today, I'll cover three main topics: packaging gems, cross-compiling C extensions, and linking shared libraries.

00:13:11.839 The first topic is the packaging of gems, which we recently enhanced by adding the new command 'rwm build' to our RubyWasm gem. This command will package gem sources into the WebAssembly binary.

00:13:38.360 The packaged gems are stored in a bundled directory within the virtual file system. You can set up the load paths using the usual 'bundle setup' command, and then require the gems as you would normally in Ruby.

00:14:04.959 The next challenge is cross-compiling C extensions to WebAssembly. To build C extensions, we must interact with 'mkmf,' which is the core of the C extension build system in Ruby. 'mkmf' stands for make makefile—it generates a makefile from extconf.rb files.

00:15:07.680 Currently, there are three significant users of 'mkmf': the Ruby built-in extension system which uses it to build extensions, the Ruby compiler that builds gems, and RubyGems itself when you execute the gem install command. However, RubyGems does not inherently support cross-compilation.

00:15:51.320 Instead, the existing Ruby build system overrides top-level constants which tell 'mkmf' that the compilation target differs from the host platform. While this works for our current C extensions in RubyGems, it doesn't cover all edge cases, particularly those involving complex configurations.

00:17:44.080 To address the limitations of current approaches, we have introduced cross-compilation support to 'mkmf' itself through the new target_ruby_config option, enabling developers to specify the appropriate Ruby configuration file for their deployment platform. This will be available in Ruby 3.4.

00:18:59.799 We can successfully cross-compile C extensions to WebAssembly. The next step is linking the compiled libraries to the Ruby interpreter, which we can achieve through two methods: static linking or dynamic linking.

00:19:46.000 Although we have previously supported static linking for built-in extensions, we opted for dynamic linking to offer better support for RubyGems due to the RubyGems ecosystem's distinct build model which makes static linking less efficient.

00:20:47.320 Dynamic linking can simplify the build process. However, historically, WebAssembly hasn't standardized dynamic linking at runtime, which could diverge from the typical compatibility of WebAssembly runtimes.

00:21:47.080 Fortunately, recent developments in the WebAssembly community, particularly with the introduction of the Component Model, promise to enhance dynamic linking capabilities. This will allow us to build a more robust linking mechanism that can compose multiple modules seamlessly.

00:23:30.839 The Ahead-Of-Time (AOT) dynamic linking mechanism we are developing is distinct from traditional runtime linking as it is handled during the build time, resulting in a self-contained binary that includes all dependencies.

00:25:31.919 We are working hard to fine-tune the final binary that is currently around 30 megabytes, so we need to implement several optimizations, including reducing the binary's size without sacrificing performance.

00:26:01.600 The first step is to eliminate any overhead caused by asynchronous control flow in WebAssembly, which will significantly impact the overall efficiency of the binary.

00:27:10.480 Overall, we have many opportunities to enhance the RubyGems support in RubyWasm. The roadmap involves continuous improvement and refinement of the interpreter's size and performance.

00:29:04.159 In summary, RubyGems support will soon be available in RubyWasm. Ruby is evolving alongside the latest WebAssembly technologies, including the component model, which will contribute to making Ruby even more WebAssembly-friendly. Let's collaborate to make Ruby's future exciting and web-friendly!

00:29:47.080 Finally, I would like to acknowledge the support from the Ruby Association grant program and express my gratitude to Matz and Koichi for their ongoing guidance. Thank you for your attention!

See Slides on speakerdeck.com

RubyKaigi 2024