Helix

Summarized using AI

Bending The Curve: Putting Rust in Ruby with Helix

Godfrey Chan and Terence Lee • September 18, 2017 • Hiroshima, Japan

In the presentation titled "Bending The Curve: Putting Rust in Ruby with Helix," held at RubyKaigi 2017, Terence Lee and Godfrey Chan discuss their open-source project Helix. Helix is an FFI (Foreign Function Interface) toolkit designed to simplify the process of integrating Rust code into Ruby applications by enabling developers to write native extensions in Rust. The presentation addresses the challenges and lessons learned from the development of Helix and highlights the benefits of combining Ruby and Rust.

Key Points Discussed:
- Background on Helix: Two years prior to this talk, a similar presentation was given, marking the initial showcase of Helix at RubyKaigi. Helix aims to overcome the performance limitations of Ruby by leveraging Rust.
- Performance Bottlenecks in Ruby: While Ruby excels in many areas, its speed can be a limiting factor in performance-critical applications such as video encoding and machine learning. Native extensions, often written in C, can enhance performance but present challenges in terms of complexity and safety.
- Rust as a Solution: Rust is introduced as a safer alternative to C. The language provides memory safety through its ownership model and allows for zero-cost abstractions, enabling high-level code to be both efficient and performant. For instance, operations in Rust can be just as concise as those in C, but with significantly improved safety guarantees.
- Helix Features: Helix streamlines the process of writing Ruby native extensions in Rust. Developers start by writing their code in Ruby and only optimize sensitive segments using Helix, minimizing the need for a complete rewrite. Helix automatically generates necessary boilerplate code, which simplifies development.
- Demo of Helix's Capabilities: The presenters provide a live demo showcasing a Rails app that transforms English text into Australian slang using Helix. The demo illustrates the seamless integration of Ruby and Rust, demonstrating how to build and test a Rust-based functionality within a Ruby application.
- Deployment and Development Challenges: The presentation discusses the complexities of deploying Helix, including the use of CI environments to build binaries and the creation of a domain-specific language (DSL) to minimize the learning curve for Ruby developers unfamiliar with C or Rust. Additionally, challenges such as type safety and Rust's macro system are addressed.
- Future Developments: The roadmap for Helix includes user feedback, potential enhancements, and the goal of making Rust native extensions more accessible to the Ruby community. The presenters encourage collaboration and improvement of Helix through the community's involvement.

Conclusions and Takeaways:

Helix offers a robust toolkit for Ruby developers looking to boost performance through Rust. By maintaining Ruby's ease of use while capitalizing on Rust’s efficiency and safety features, Helix has potential for wide application in projects requiring high performance. The presentation wraps up with an invitation for community input and further dialogue on the toolkit's future advancements.

Ultimately, this session at RubyKaigi 2017 showcases the promising intersection of Ruby and Rust, encouraging developers to explore Helix and contribute to its evolution.

Bending The Curve: Putting Rust in Ruby with Helix
Godfrey Chan and Terence Lee • September 18, 2017 • Hiroshima, Japan

RubyKaigi2017
http://rubykaigi.org/2017/presentations/chancancode.html

Two years ago at RubyKaigi, we demonstrated our initial work on Helix, an FFI toolkit that makes it easy for anyone to write Ruby native extensions in Rust. In this talk, we will focus on the challenges and lessons we learned while developing Helix. What did it take to fuse the two languages and still be able to take advantage of their unique features and benefits? How do we distribute the extensions to our end-users? Let's find out!

RubyKaigi 2017

00:00:12.019 I was introduced as someone who speaks Japanese, so let me apologize for my poor skills.
00:00:22.220 I have some blue hat stickers, so if anyone is interested in talking to me afterwards, please come find me.
00:00:30.449 That's it for the greetings.
00:00:36.140 Hello, I'm Terence. I go by @hone0-2 on Twitter, mostly known for my hats. I work for Heroku, and like Conan said, I am here to talk about my work.
00:00:55.440 In case you missed it, this is a picture of me. I used to be on the Rails core team until relatively recently, and this is how we could tell us apart.
00:01:10.979 As for my current job, I work at Tilde in Portland, Oregon, the same city as Joanne. You might know us from our product called Skylight, which is a Rails performance monitoring tool.
00:01:22.200 Now about our travel from the United States here—it’s quite a long way to Japan. Speaking of the U.S., we are also known for our delicious food.
00:01:43.800 Fusion cuisine is particularly interesting. For instance, here is what sushi looks like in America. You might be asking, who puts that much sauce on sushi? Well, we actually have options to put extra on.
00:01:59.729 The reason I find fusion cuisine compelling is that as a chef, one must carefully balance between creating fusion and confusion in the culinary world.
00:02:24.590 On that note, if you get the chance to visit America, I recommend trying a variety of dishes.
00:02:55.069 Today, we are here to talk about Helix, an open-source project we created to facilitate writing Rust code in Ruby.
00:03:01.100 To understand why this is a good idea, let me share some backstory. I apologize to those who heard this before; two years ago, I gave a similar talk at RubyKaigi with my coworker.
00:03:30.659 Unfortunately, he recently became a parent and couldn't join me this time. If you wish to send your greetings, you can find him online.
00:03:51.709 Moving back to Helix, I believe everyone here shares a love for writing Ruby. While Ruby is a fantastic language, we also know that it can sometimes be slow.
00:04:14.780 Most applications experience IO-bound workloads where the speed of the programming language becomes less significant. Often, we find ourselves waiting for database queries or HTTP requests.
00:04:57.290 However, there are scenarios where performance is paramount, such as video encoding or machine learning, where Ruby's slowness may hinder progress.
00:05:14.500 In Ruby, we have a well-known solution for this issue. By writing native extensions, you can achieve better performance. For example, when you require the JSON gem, you aremost likely using a C-based implementation.
00:05:41.280 This underlying native implementation can be faster, but from the user's perspective, it operates in a completely transparent manner.
00:06:02.010 The Ruby core team uses this same concept for classes like Date or Pathname to ensure they perform as efficiently as possible.
00:06:36.310 An interesting case of this phenomenon occurred a few years ago, when Sam Saffron discovered that the String#blank? method was a hotspot for performance issues.
00:07:01.629 He wrote a C extension called Fast Blank, which made this operation 20 times faster with only 50 lines of code.
00:07:40.840 We should then consider why we're not doing this more frequently. Simply put, the problem is C. Ruby programmers primarily work in Ruby, not C.
00:08:05.000 C programming can be risky and unsafe, and it increases maintenance burdens and the barrier to contributions. It requires a careful understanding of pointers and other complex details.
00:08:36.990 At Skylight, we encountered a similar problem, as we were developing a performance monitoring tool. Initially, we wrote our agent in Ruby, which worked fine.
00:09:06.820 However, we soon faced limits to what we could achieve in Ruby and realized we needed a native extension, likely in C or C++. But we found this to be a cumbersome solution.
00:09:43.640 Ultimately, we turned to Rust. I’ll let Terence explain why Rust is an effective option for tasks like these.
00:10:22.430 Not everyone here may be familiar with Rust, but it's a systems programming language, similar to C.
00:10:41.800 It compiles to quick assembly code with no runtime overhead, unlike Ruby. Rust also features an advanced type system, enabling it to encapsulate complicated use cases.
00:10:54.400 One fascinating aspect of Rust is that if your code compiles, it’s guaranteed not to crash. This allows us to provide similar memory guarantees within Ruby, but without a garbage collector.
00:11:08.590 This guarantees memory safety in Rust thanks to its ownership model. Another significant feature of Rust is the idea of zero-cost abstractions.
00:11:44.300 While programming in Ruby, there is often a tension between writing higher-level abstraction code and lower-level code that may perform better. Eric's 2014 talk highlighted this issue.
00:12:19.020 However, in Rust, you don’t have to make these trade-offs because the compiler handles it for you. One example is summing numbers in an array.
00:12:38.560 In Rust, you can use a higher-level function like fold, similar to Ruby's inject, and it’s actually faster because you’re letting the compiler perform optimizations.
00:13:29.190 Turning back to the Fast Blank example, this shows how concise Rust can be. With about 12 lines of Rust code, we can achieve the same result as 50 lines of C.
00:14:10.790 Helix aims to eliminate much of the boilerplate code needed for these extensions.
00:14:32.380 You should always start writing your code in Ruby and only shift performance-sensitive portions to Helix as necessary. This way, you won't be forced to rewrite your entire app.
00:15:27.680 Next, we have a demo prepared. Godfrey did an excellent telecast, and everything you need to know is at this URL.
00:15:49.320 In this demo, we are showcasing a simple text transformer Rails app. It takes some English text, and with a click, it transforms it into Australian slang.
00:16:23.240 Helix provides generators, which allow you to create special files specific to Helix. You get both a Ruby gem and a Rust crate, along with a Cargo.toml file.
00:16:58.410 The Cargo.toml file is where you specify your dependencies, while lib.rs acts as the main entry point for your Rust code.
00:17:20.360 Here’s an example of a simple Hello World output generated automatically by Helix.
00:17:43.430 In our Rails app, we can test the Helix crate as if it were regular Ruby code. For our first test, it's failing because we haven't implemented the 'flip' method yet.
00:18:05.289 We’ll replace the hello method with the flip method that takes an input string and flips certain characters upside down.
00:18:30.200 In Rust, we can call high-level methods similar to Ruby, mapping characters together. Once we have flipped every relevant character, we return the result to the Ruby side.
00:19:08.530 We then run our tests again, and this time they should pass. Since flipping isn't complete without table-flipping, we will add tests for that as well.
00:19:53.920 After implementing these tests, running them should demonstrate that we now have a functioning Helix crate usable within our Rails app.
00:20:23.870 This showcases how we can deploy it using Heroku. To handle this, we utilize a Rust build pack alongside the Ruby build pack.
00:20:43.360 Upon deployment, the Rust build pack runs first, setting up the Rust toolchain, before proceeding with the usual Ruby installation steps.
00:21:09.890 So if you access hilux-flipping.com, it should display the results as expected.
00:21:40.860 Now that we have covered the demo, let's move on to implementation details of Helix.
00:22:06.350 I want to highlight some of the challenges we faced while building Helix, particularly in making it easy to use.
00:22:39.370 The Ruby C API differs significantly from the Ruby programmers' experience, which raises the barrier for Ruby developers not familiar with C.
00:23:09.200 To overcome this, we designed a domain-specific language (DSL) to make using Helix straightforward.
00:23:38.150 The rail user experience should feel familiar, with a reasonable number of lines of code translating to equivalent functionality.
00:24:10.300 To facilitate the Helix design, we opted against writing our own parser, instead leveraging Rust’s powerful macro system.
00:25:03.850 Macros in Rust operate on parsed tokens rather than raw text, making the parsing process more manageable.
00:25:36.630 As we create new struct types and method definitions, we can generate the associated boilerplate efficiently.
00:25:57.640 However, Rust's macros come with certain limitations, such as having a recursion limit and requiring valid Rust syntax after expansion.
00:26:25.410 To address recursion issues, we simply increased the limit, which allowed for greater flexibility in implementation.
00:27:06.090 The requirement for valid syntax often made it challenging to use macros for partial results, but we were able to push macros into their own computations or expansions.
00:27:48.200 The challenge of type safety is critical as well, because the Ruby C API uses void pointers extensively, which could lead to type-related errors.
00:28:10.300 Rust provides strong guarantees against these types of issues, offering better maintainability and easier debugging for users.
00:28:48.420 Transitioning to full Rust managed types allows for cleaner code without assumptions about value types, ultimately leading to more robust implementations.
00:29:25.111 Finally, on deployment, we examined options such as cross-compilation but decided to use CI environments to build our binaries, letting users easily access them.
00:30:05.360 The roadmap for Helix defines current goals and future ambitions, which you can check out on our website.
00:30:45.990 As we wrap up, your feedback on Helix is invaluable. Even if you cannot implement everything right now, we welcome early input.
00:31:03.780 If you're interested in helping, consider joining our team for debugging or documentation tasks.
00:31:27.520 If you're keen on crafting Rust native extensions, Tilde can assist you.
00:31:53.780 That wraps up my presentation. Thank you so much for your attention.
00:32:19.200 If you wish to try the text transform gem, you can install it directly without a Rust compiler due to our CI setup.
00:32:45.000 Do we have any questions? Now is the moment.
00:33:08.700 If there are none, we will be around to chat more. If you try Helix and have feedback or questions, please share.
00:33:41.090 One question—have you considered developing a macro to do Ruby-style string interpolation?
00:34:12.760 Currently, with Rust's macro system, strings are parsed as single tokens, which limits parsing capabilities for such features.
00:34:39.180 In the future, procedural macros may enable such functionality, providing developers more flexibility without complexity.
00:35:05.260 Thank you for the engaging questions. Rust's maintainability is taken seriously. They focus on stability and compatibility.
00:35:30.420 Having reached Rust 1.0, this stability allows developers to trust that their code will remain functional across versions.
00:35:47.230 New features introduced in Rust are generally backward compatible, meaning only developers using private APIs may face issues.
00:36:05.380 Should you experience breakage due to compiler changes, I encourage you to report it.
00:36:24.890 Thank you! We appreciate your interest in us.
Explore all talks recorded at RubyKaigi 2017
+59