From Interpreting C Extensions to Compiling Them

Talks

Benoit Daloze

#just-in-time-compilation

#ruby-implementations

From Interpreting C Extensions to Compiling Them

by Benoit Daloze

In the presentation titled "From Interpreting C Extensions to Compiling Them," Benoit Daloze explores the evolution and implementation of C extensions within the Truffle Ruby framework. He begins by outlining the reasons for the existence of C extensions in Ruby, primarily for binding to existing C libraries and improving performance. While he notes that the usage of C extensions has decreased, he highlights that a significant portion of the Ruby ecosystem, 43% of the top 10,000 gems, relies on native extensions. Daloze discusses the journey of implementing these extensions starting from using Truffle C to the introduction of Sulong, an LLVM bitcode-based system for efficient runtime behavior.

Key points from the talk include:

- Introduction to Truffle Ruby: A high-performance Ruby implementation utilizing the Graal JIT compiler and targeting compatibility with C extensions.

- Purpose of C Extensions: C extensions bind Ruby with C libraries and enhance performance, though their necessity is debated.

- Native Extension Usage: Only 4% of the top gems have C extensions, 46% depend on them transitively, indicating their importance for compatibility.

- Implementation Approaches: Daloze discusses various methods, emphasizing the dual benefits of using the Truffle framework for development.

- Transition to Native Compilation: He explains that the recent shift from JIT compiling to native compilation allows for faster execution and better handling of large extensions.

- Challenges with Memory Management: The talk addresses the complexity of managing memory representations in C extensions and highlights strategies for efficient interaction between Ruby and C.

- Future Developments: Daloze concludes with ongoing explorations of strategies to enhance the flexibility of C extensions in Ruby.

In conclusion, the talk emphasizes that while the Ruby extension API has its challenges, recent advancements in compiling and managing C extensions enhance the overall functionality and performance of Ruby applications, demonstrating the significant reliance on native extensions within the Ruby community. Daloze encourages experimentation with Truffle Ruby in its recent releases.

00:00:03.719 Okay, hello and welcome to my talk.

00:00:11.000 Today, we are going to discuss interpreting C extensions and then turning them into compiling.

00:00:17.039 My name is Benoit Daloze. I work at Oracle Labs in Zurich and have been involved in Truffle Ruby since 2014.

00:00:22.119 I hold a PhD in Dynamic Languages and I am the maintainer of Ruby spec and set up Ruby. I am also a committer for Truffle Ruby.

00:00:28.400 Truffle Ruby is a high-performance Ruby implementation that uses the Graal JIT compiler to target full compatibility with C 3.2, including C extensions.

00:00:40.360 As a point of reference, applications like Mastodon and Discourse can run on Truffle Ruby due to the high level of compatibility.

00:00:46.160 You can find Truffle Ruby on GitHub and so on.

00:00:52.199 Now, let's look into why Ruby has C extensions in the first place. There are two main reasons for this.

00:01:04.600 The first reason is to bind to existing C libraries. For instance, libraries like OpenSSL, MySQL, and PG can all be linked via corresponding native libraries.

00:01:17.159 That's a common reason for using C extensions. The alternative, which is typical, is to use the FFI gem. However, only a small proportion of gems utilize the FFI gem.

00:01:29.600 Furthermore, the FFI gem struggles when libraries have numerous macros in their headers, as one would need to replicate extensive logic in Ruby code.

00:01:40.520 The second purpose of C extensions is to improve performance. Sometimes Ruby code can be too slow.

00:01:52.439 For instance, both JSON and MessagePack have C extensions to speed up serialization. However, this practice has become less encouraged over the years.

00:02:04.759 As discussed in Maxim's talk, it's seen as less necessary now, and Truffle Ruby executes Ruby code so swiftly that writing a C extension may not have significant advantages.

00:02:10.800 Moreover, C extensions usually require higher maintenance.

00:02:16.800 So far, I have talked about C extensions, but what I really mean is native extensions, as they are not limited to C; you can also use C++ or Rust and other languages.

00:02:24.959 If we look at usage, we can count among the top 10,000 gems by the number of downloads and see how many of them actually use a C extension.

00:02:31.800 Only 4% of these gems have a C extension, which may indicate that C extensions are not that crucial for compatibility. However, if we account for gems that have a transitive dependency on a gem with a native extension, the percentage increases significantly.

00:02:42.800 For the top 10,000 gems, it's 46% that depend transitively on a C extension.

00:02:48.840 It's important to note that we don't count FFI in this statistic, as FFI is also treated as a C extension, but in other Ruby implementations, it isn't.

00:02:54.360 We can also be more precise; for instance, the JSON and Rack gems have a probable fallback, allowing them to work even if the extension is not supported.

00:03:01.080 Even with this, it still amounts to 43%, meaning that a large portion of the Ruby ecosystem relies on some extensions, making it imperative to support them for compatibility.

00:03:07.920 Now, for JRuby, it's a bit more complicated due to Java extensions, and I will not dive into that detail here.

00:03:20.680 If we examine the compatibility, of course, C defines the specs, ensuring compliance, while Truffle Ruby passes about 98% of the CPI specs.

00:03:28.000 This level of compatibility is commendable. However, JRuby does not implement C extensions—at least not in the current version—so it does not pass these tests.

00:03:41.239 Next, I'd like to delve into how we can implement C extensions.

00:03:52.959 For the past ten years, we have been keen on implementing these extensions as they are crucial for compatibility. We have tried various approaches, each with its own set of pros and cons.

00:04:04.360 As a bit of background, let me explain what Truffle is. Truffle is a framework designed to facilitate the creation of high-performance languages.

00:04:09.840 The main advantage of writing an interpreter with the Truffle framework is that you essentially get a JIT compiler for free. This works because the Graal VM compiler, a language-agnostic JIT compiler, can partially evaluate any language's bytecode and generate efficient machine code from it.

00:04:28.000 For example, suppose I have a Ruby method or block. The JIT compiler, which predominantly understands Java, interacts with the Truffle Ruby interpreter to identify operations, such as a plus operation, and JIT compiles to machine code.

00:04:46.840 The end result is machine code optimized for that specific Ruby method or block. Essentially, it is akin to having written a JIT compiler specifically for Ruby, but without the overhead of doing so.

00:05:03.280 This dual functionality of not needing to duplicate logic between the interpreter and the JIT compiler is a common pain point that many virtual machines suffer, leading to bugs and performance issues.

00:05:14.720 The journey began in 2014 with an initial prototype, utilizing Truffle C. Truffle C, as its name suggests, is a Truffle interpreter for the C language.

00:05:38.080 This approach was quite innovative and unusual, as C isn't conventionally viewed as an interpreted language.

00:05:45.880 You might wonder why we took this route instead of simply using Clang.

00:05:52.120 I will answer this question in more detail, but in summary, we aim for more freedom of representation. For instance, we can represent a C struct as a Ruby object.

00:06:05.400 This seems counterintuitive, as a C struct is merely a layout in memory, while a Ruby object is more complex. However, this approach enables us to overcome limitations.

00:06:17.880 Moreover, surprisingly, performance benefits can arise because a Graal VM JIT compiler can compile both C and Ruby together.

00:06:29.360 Thus, it can inline functions across both languages. When we compile code, we can optimize a Ruby method that calls a C function, which subsequently calls another Ruby method.

00:06:41.720 Inlining is crucial for performance, as shown by the early statistics, which revealed that calling C extensions with inlining led to a significant performance gain.

00:06:54.400 When C calls Ruby, it often results in slowdowns; however, inlining mitigates this problem.

00:07:04.320 Two years later, we see the inception of Sulong, a successor to Truffle C that allows for passing C code more efficiently. Instead of passing C directly, Sulong employs LLVM bitcode, which serves as an intermediate representation.

00:07:50.760 This approach is faster than typical C passing, as it allows us to compile C to bitcode ahead of time with Clang. Sulong interprets this bitcode, allowing for efficient runtime behavior.

00:08:02.000 Now, transitioning to native extension scenarios, we initially had challenges with representing structures directly.

00:08:15.400 In C, Ruby headers define value types commonly as unsigned long. This isn't convenient, as objects in Truffle Ruby, represented as Java objects, cannot have direct memory addresses.

00:08:27.280 Java's garbage collector can move these objects at any time, so obtaining valid addresses isn’t practical, as any address you might obtain could be invalid.

00:08:40.199 Sulong's solution provides a method for managing representations creatively. For example, when defining a method in a C extension taking a Ruby object, we manipulate object fields directly without converting these objects to integers.

00:08:53.600 Let’s consider redirecting access to a C struct, where commonly we would use Ruby macros to reference Ruby objects directly.

00:09:07.760 Instead of reading memory addresses, our implementations can return Ruby objects directly, streamlining the interaction.

00:09:19.160 Instead of adhering to traditional memory access patterns, we facilitate interoperability, allowing for Ruby and C exchanges while maintaining efficiency.

00:09:31.640 However, when working with inter-library calls, unexpected leakage of Ruby objects into native memory can complicate matters.

00:09:44.320 For instance, when interfacing with libraries like libuv, Ruby objects could be passed around leading to further complications.

00:09:52.520 Inspired by these observations, we introduced handles, which allow us to associate native pointers with Ruby objects and prevent any unnecessary overhead.

00:10:05.400 While handles facilitate this representation change, we aim to minimize their use to situations where they are absolutely necessary.

00:10:20.160 This year, I made significant changes to Truffle Ruby, allowing it to handle extensions natively, moving away from reliance on Sulong.

00:10:34.000 This transition provides various advantages, including faster startup and warm-up times, eliminating the costly JIT compilation step.

00:10:46.760 By leveraging native compilation tools like Clang or GCC, we foster more trustworthy code execution.

00:10:56.320 With direct compilation, large extensions such as gRPC, which often utilize advanced C++ features and edge cases, now become feasible, showcasing our resolve.

00:11:12.960 That said, this new approach has drawbacks; we no longer have the flexibility to perform certain optimizations or inline caches.

00:11:23.560 For instance, every variable must adhere to type specifications in C, and we cannot afford the same level of dynamic behavior previously enjoyed.

00:11:38.840 Similarly, the constraints of inline caches—previously beneficial for performance—must now be adapted to suit native specifications.

00:11:50.720 In focusing on structures, we needed to rethink how to manage specific Ruby C structs. For instance, instead of relying on RBASIC, which would have required instantiating traditional C structures, we applied macros to interface directly with desired Ruby classes.

00:12:05.680 One such example arises with IO objects, where we also construct a pointing mechanism to facilitate access to the relevant C fields.

00:12:17.920 Moreover, the challenge involves bulk structures, like RB Encoding, part of Ruby's core, demanding specialized consideration during implementation.

00:12:31.000 Additionally, the use of inline caches has resurfaced as a notable tactic traditionally used in the Ruby environment.

00:12:45.960 Despite needing a redesign, this leveraging of caches along with the existing C methodology aligns neatly with optimal performance operational principles.

00:12:59.640 In summary, we have seen the evolution from interpreting and JIT compiling C code to running everything natively through a customizable toolchain.

00:13:14.160 This enables extensions to be utilized effectively while preserving valuable instructions found within the standard libraries.

00:13:27.720 While the extension API still has its pitfalls—it’s immensely better than the CPython API, which exposes a significant number of structures.

00:13:39.800 Many developers realize that mutable design choices can contribute to stunted progress and performance.

00:13:45.240 Therefore, ongoing research continues to explore strategies that combine the intuitiveness of C extensions with the flexibility of Ruby.

00:13:58.520 In conclusion, around 43% of the top 10,000 gems are dependent on gems with native extensions, underscoring the importance of supporting them for compatibility.

00:14:05.000 The Ruby extension API is flexible and can be implemented by alternative Ruby implementations.

00:14:12.520 Truffle Ruby initially approached it by JIT compiling C extensions, which provided many advantages.

00:14:19.560 We have now transitioned to running these natively, which yields different yet significant advantages.

00:14:26.680 Overall, the challenges faced by C extensions revolve around managing memory representation effectively.

00:14:32.760 With advancements in handling functional macros, we can refine and improve efficiencies further.

00:14:39.000 If any of you would like to experiment with Truffle Ruby, recent releases are available along with early access development builds.

00:14:45.520 Thank you for listening!

See Slides on eregon.me

RubyKaigi 2024