Ruby's Core Gem

00:00:00.000 Ready for takeoff.

00:00:17.340 Thank you all for coming. I'll get cracking. Today, I want to talk to you about an idea that's been around for a while, which we can call Ruby's core gem.

00:00:20.520 I'm a big fan of giving the main idea up front to get straight into what we're trying to achieve. Ruby has a core library; it consists of classes like Array, Hash, and String. Currently, this core library is implemented in C in the standard version of Ruby. For example, here’s the code to implement 'loop do,' which is a core library routine. You can see it’s written in C and has to be broken apart into a couple of methods because that’s how things work in the C version of Ruby. This C code isn't very readable; it's hard for us to understand as application programmers.

00:00:42.060 Interestingly, it's also challenging for the Ruby VM to understand and do anything meaningful with it. So, the big idea is to rewrite this core library in Ruby, using the language we use for our applications. Let's use it for the core library as well, and you can already see some benefits here. This code is much more understandable. If you want to know what 'loop do' does, you can simply read this Ruby code. You’ll see that it runs a while loop and yields the block in each iteration. Moreover, you might notice things you didn’t know before. For instance, did you know that if you call 'loop' without a block, it gives you an infinite enumerator? You can figure that out just by looking at the source code, even if you couldn’t see it from the C code as easily.

00:01:11.040 We can also determine if we can break out of a 'loop do.' Yes, you can, by using the StopIteration exception. That's also clear from the Ruby code. It turns out this approach has many benefits, not only in terms of understandability but also regarding how the VM can optimize it and how we can use tooling on it. The discussion about re-implementing Ruby's core library in Ruby also opens a conversation about the potential future of Ruby and how we can enhance it in the long term.

00:02:02.760 A bit of context about me: I'm from Cheshire in the UK—that's Cheshire as in the cat. I have a PhD in compiling Ruby and founded TruffleRuby, which is an alternative implementation of Ruby. I'm using this as an example for some of the work I’ll discuss today. I was formerly at Oracle Labs and am now at Shopify, which is a really supportive workplace with great people. My interest lies specifically in optimizing idiomatic Ruby code, so I focus on optimizing Ruby as it is rather than transforming Ruby into something else to achieve optimization.

00:02:10.920 Interestingly, I also lead a British Cavalry Squadron in my spare time and am keen to meet other Ruby reservists and veterans, if there are any out there. One of the core concepts I want to talk about is Ruby's Tower of Libraries. We can discuss the various libraries Ruby has and where they sit in a hierarchy.

00:02:37.620 At the bottom, we have the language, the core Ruby language that we have in the Ruby interpreter. The core libraries exist one level above that, and then above that, we have the standard library, which includes things like JSON that you can require without installing anything. Further up, we can talk about gems and user code. The bottom layer can consist of Ruby and C code; the further down you go, the more code is written in low-level C, while the higher up it is, the more is written in Ruby. Currently, the core library is almost entirely implemented in C, and that’s what we’ll delve into today.

00:03:00.300 At the bottom, we have the Ruby language itself, which provides a very small subset of functionalities—classes, modules, methods, method calls, and some control structures such as if, while, case, and others. However, this subset is quite brief; very little else is provided by the language. For instance, in this code example, we've got an if statement and a method call, which is mostly what the language provides.

00:03:30.000 Next, at the core library level, we have things like array and hash but also lower-level items like numbers and strings. Interestingly, numbers and strings are actually part of the core library; they're not just provided by the language. We also have some control structures like loop and array.each, and hash.each, which are provided by the core library, despite being related to control flow.

00:03:51.360 The core library is automatically available; you don’t need to require it, as it's always ready to use. It's implemented as a C extension built into Ruby, but it uses the same API. There are approximately 2250 methods in this core library, making it quite large, so there's a lot included in Ruby's batteries. For example, if we have something like a hash and we call methods like '.values.sort.first,' 'add,' and 'values sort first,' these are all provided by the core library.

00:04:15.660 Above the core library is what we call the standard library. This needs to be required in Ruby, with some exceptions, but it is available without installing anything. It’s part of the Ruby distribution, but we won’t focus too much on the standard library in this talk since it’s not particularly relevant. However, for instance, the 'json.generate' method illustrates a feature of the standard library.

00:04:36.960 An interesting point about the standard library is that it is being 'gemified,' meaning that over time, it is becoming a gem that can be installed separately if desired. Above this, we find user gems and application code, which are Ruby code loaded at runtime from outside the interpreter and outside the Ruby distribution. This code can come from a gem or from your code repository. This distinction matters to us as programmers because we tend to view gems as separate from user code, but for the VM, it doesn't really make any difference; it's all code loaded from disk at runtime.

00:05:08.520 For example, some Rails code in a controller would be considered user code. Sometimes, gems and user code are written in C as well—gems like Nokogiri or OpenSSL contain substantial amounts of C code. So, while we can include C code, it still requires loading at runtime. There are many positive aspects of the core library as it stands—it's always available, meaning you can't end up with the wrong version or find yourself without the core installed. It can be used to build bigger things because it's always accessible.

00:05:39.900 For instance, Ruby gems requiring core libraries are available as soon as the interpreter starts. This readiness is excellent for application boot times. The core library can utilize VM internals to execute tasks that aren't feasible in pure Ruby. For example, low-level file I/O cannot be accomplished directly in Ruby, but it can be implemented as part of the core library using a file object. I'll explain this further because it's crucial to our discussion.

00:06:02.340 However, there are downsides to the core library as well: it's far too large, with over 2000 methods, making it difficult for us as VM implementers to understand and work with. There's no Ruby code that you can read to grasp what a method does; once you delve into it, you find yourself in code that is not always comprehensible, even if you do understand C. You cannot step into the core code and see what it’s doing, nor do you have Ruby code to utilize profiling or coverage tools—it's all C extension code.

00:06:39.300 Another drawback of C extension code is that, perhaps surprisingly, C code can sometimes perform worse than Ruby code. We'll explain more about why this is the case as the Ruby programming language evolves. Could we achieve the best of both worlds? Could we realize the benefits of Ruby implementation while minimizing the disadvantages? What we're proposing is to take that core library, split it up into two parts: one part we'll call the new core library, reimplemented in Ruby, and the other part a smaller set of primitives, which would be implemented as they currently are in C or Java, as seen in implementations like JRuby or TruffleRuby.

00:07:08.520 This split should give us the best of both worlds, with the bulk of our code existing in Ruby, which Ruby developers can read, understand, and debug. This would also mean that the VM can better optimize Ruby code. We're developing tools like YJIT to enhance the optimization of Ruby. Additionally, we would retain a small, well-defined set of underlying primitives that we can specifically teach the compiler about since there will be fewer of them. This enables the VM to work effectively and comprehensively understand their function.

00:07:37.080 Ruby implementations are already starting to explore this technique. MRI, also known as CRuby, does it to a limited extent; JRuby does it a bit more, and TruffleRuby engages with these ideas more extensively. We’ll discuss Rubinius later, as it pioneered this technique. Now let’s consider how MRI, or CRuby, currently implements this to a small extent. This example from the MRI source code showcases a very simple method called 'tap.' This method allows you to run a block with a value and return that value, making it useful for injecting values into a pipeline of method calls.

00:08:20.280 This simple behavior is evident in the pure Ruby code provided in MRI. The kernel module is written in pure Ruby code and the 'tap' method yields self and then returns self. This makes it easy to understand. Now, consider a more complex example like 'frozen.' This method determines if an object is frozen, which is a method included in the kernel that all objects inherit. However, reading the frozen status demands accessing C code to define the frozen condition.

00:08:55.980 MRI includes a construct for embedding C code within Ruby. This enables lower-level functions that cannot be performed in pure Ruby to be implemented. Therefore, we would run C code to invoke the C extension function RB_OBJECT_FROZEN, which checks if the object is frozen. Such implementation allows for a critical optimization layer while relying on a small amount of C code that can execute efficiently.

00:09:24.480 MRI has 2194 core methods implemented in C; the vast majority exists as C implementations. However, only 64 core primitives—distinct special methods—are implemented in C. There are only 31 instances of inline C, which means while it’s a promising concept, it’s not widely adopted. Currently, there are seven optimized core methods that exist, while only 101 core methods are implemented in Ruby, showing that work is just beginning in re-implementing core methods in Ruby.

00:09:54.360 TruffleRuby takes this concept quite a bit further, as it implements core methods in pure Ruby. Notably, it also has a 'tap' method, which mirrors MRI’s implementation. Here, you can appreciate one of the advantages: the code is consistent across implementations, meaning MRI, JRuby, and TruffleRuby could all potentially share this code. Yet, TruffleRuby extends this further; its 'hash' class, for example, has routines like 'key,' which returns the key for a value, and 'to_a,' which gives an array representation of key-value pairs in the hash.

00:10:18.960 Implementing 'key' involves using 'each_pair' to look for a match, returning the corresponding key or nil accordingly. In the same way, for 'to_a,' we can generate an array and use 'each_pair' to populate it with each key-value pair from the hash. This illustrates how a single primitive, 'each_pair,' can facilitate two methods of Ruby by acting as a foundation for their implementation.

00:10:46.440 In TruffleRuby, there are 611 core methods implemented in Java and 353 core primitives also in Java. We have 2386 total core methods implemented in Ruby, indicating a significant shift towards Ruby-based implementation away from C. Although there may be confusion about why this doesn't tally up to 2250, the discrepancy arises due to helper methods and similar entities that complicate the simplicity of counting—all that matters is that the majority of functions are now implemented in Ruby instead of Java.

00:11:22.920 JRuby also employs similar techniques, though we won’t delve into it too much. In JRuby, you might find the 'integer times' routine, which follows a simple implementation utilizing a while loop and yield, revealing its Ruby-centric optimization capabilities. Why focus on Ruby implementations for core methods? The foremost advantage of writing them in Ruby is increased understandability—Ruby developers can easily study and comprehend the code directly, enabling self-answers to questions regarding core method functionalities.

00:12:06.600 When core library routines are in Ruby, they foster clarity and consistency, supported by Ruby programmers' shared standards. Additionally, we can leverage familiar tools like debuggers, profilers, and code analyzers instead of facing an impenetrable black box written in C. This facilitates optimization and agility in contributions to the core library.

00:12:37.560 Moreover, if we can reformulate how core libraries function and how implementation can merge across benchmarks like MRI, TruffleRuby, JRuby, Artichoke, and any future interpretations, we stand to benefit by building an even more robust library that integrates improvements from varied platforms. Once again, we could let the underlying VM focus on the primitive while the broader development community takes charge of enhancing the core library. In doing so, developers can contribute based on firsthand knowledge from their application experiences.

00:13:11.640 A surprising benefit of transitioning to a Ruby core library could be optimization. As we’ve established, C code can occasionally exhibit slower performance than Ruby code. By writing core methods in Ruby, we can wield existing Ruby optimization techniques, specializing methods for specific tasks as evidenced by some methods optimizing string comparison, which increases performance.

00:13:45.960 While we anticipate advantages, transitioning to a Ruby core library could present challenges in various respects. The first involves overhead time required during the startup phase. For instance, if TruffleRuby has 2000 Ruby methods, they must all load into the interpreter, adding to your application's bulk and increasing the initialization delay. We've noted that optimizations can only begin adapting after the application is running, so this results in slower initial performance until rapidly increasing efficiency over time.

00:14:20.880 Some developers have already had to do things like disable Ruby gems to lighten startup times, particularly for command-line tools like Ruby format by Fable, leading to useful optimizations for startup time, which would otherwise deteriorate with a shift toward Ruby core libraries. However, I believe there are workable solutions. MRI embeds YARV bytecode directly within an executable, removing the need for running the parser and simply loading it—TruffleRuby takes this concept even further.

00:14:51.840 TruffleRuby goes beyond just embedding bytecode; it embeds the objects generated from parsed Ruby code directly into the executable. This means it can start up quicker than MRI in certain situations. This perspective is quite counterintuitive; you might expect that Ruby code would inherently drag down speed or slow start but surprisingly, it can even yield quicker performance under these considerations. Another disadvantage of this approach leads us to memory performance.

00:15:31.560 While Ruby code appears more compact on the surface, it occupies more memory compared to compiled C code, which is generally much more compact. Strategies like profiling inlining, while enhancing Ruby's speed, may also contribute to elevated memory usage. In fact, any JIT compiler techniques we apply increase memory consumption too. Therefore, balancing out remaining overhead of memory will remain an open-ended problem for future consideration.

00:16:02.400 Could we mitigate some of these memory usage setbacks? Although I don’t have definitive answers on the mitigation strip, insights into potential systems to manage this would be quite welcomed as we advance. Despite the memory usage concerns manifesting per process rather than globally across users, addressing memory limitations nonetheless proves delicate.

00:16:28.440 In addition, we could employ an alternative strategy. For instance, TruffleRuby utilizes Sulong—a C interpreter designed to run C code dynamically. Although it sounds counterintuitive, interpreting compiled C code bridges a noteworthy interface, enabling us to optimize C code —like that RB_EQUAL call we mentioned earlier—using notable strategies similar to those for Ruby.

00:17:02.520 For a brief practical demonstration, I’ll share a routine named Foo, accepting a hash and value while invoking the key routine we explored earlier. Within this example, a hash includes a key-value pair 'a' to '14', and you would look up '14' to return its symbol, 'a.' The approach triggers a loop to prompt just-in-time optimization and compiles for the given arguments.

00:17:38.520 Furthermore, we can ask TruffleRuby to elucidate how it's optimizing this scenario, particularly focusing on what methods it inlines. Inlining occurs when one method integrates into another dynamically, creating a unified structure that optimizes effectively. In our case, we see that TruffleRuby inlines both 'foo' and 'hash key.'

00:18:14.700 The reason it can utilize inlining here stems from the simple nature of Ruby source code itself—uncomplicated Ruby optimizations allow us to integrate the specific methods, and it extends to primitives like 'each pair.' The fewer the primitives, the better suited we are to teach the compiler how to optimize and inline their functions seamlessly with Ruby code. This can also translate positively into YJIT and similar systems.

00:19:02.520 This functions synergistically with the premise that each pair takes a block of Ruby code and can inline it back into the Ruby structure. Thus, the combined affluence of Ruby insights inspires optimized Ruby code, with notable performance benefits arising from this approach.

00:19:35.520 These advantages extend well beyond mere understandability; we can anticipate substantive performance gains from Ruby libraries. I employ a data structure called a graph to articulate what compilers do at lower levels. By representing operations as a flowchart, we observe control flow via red arrows, and data flows through green arrows. What we’re witnessing is the return value flowing from a loaded hash index.

00:20:13.920 This visual illustrates that by passing all code through user code and integrating the core library into Ruby, we merge commands from optimized primitives initialized at a low level. We effectively compile it down to a single operation, getting the right value from the hash efficiently. This achievement is only made feasible through Ruby's core library's comprehensive synthesis.

00:20:37.200 I see this as a potential way forward for Ruby, shifting the majority of core functions into Ruby while retaining a smaller, well-defined set of primitives. This direction could cultivate a new, more manageable version of Ruby core that Rubyists can easily understand, akin to how programming languages like Haskell utilize Haskell core—a smaller, simpler design wherein all constructs can fit into your mental architecture effectively.

00:21:07.320 TruffleRuby serves as a foundational prototype for advancing this core implementation. We could develop compilers and static analysis tools that expand upon these primitives, allowing Ruby to benefit from an agile, manageable structure while promoting a more accessible codebase overall. Application developers don’t have to worry; the functionality will be unaffected, preserving all interaction as expected.

00:21:41.400 Does it need to be a gem? I’ve named it the Ruby core gem, but it doesn't have to be. It could instead be bundled within the standard Ruby version, but the option to install updates may still be viable as it evolves. I'd like to credit Rubinius, where much of TruffleRuby's core library originated. It stems from earlier Ruby implementations and has been maintained by us, the TruffleRuby team, for the past few years—a continuation of exceptional work by Evan Phoenix, Shira, and many others.

00:22:02.640 A radical thought emerges: Ruby has an extension API allowing for C extensions that are predominantly written in C. What if we could also implement these C library routines using Ruby? TruffleRuby is already working on this. We've developed a core C library routine known as 'RB_STR_NEW_FROZEN,' and we're implementing its equivalent in Ruby as well.

00:22:35.640 'RB_STR_NEW_FROZEN' checks if a value is already frozen, returning it if it is; otherwise, it duplicates the value and freezes the new copy. This accessibility reinforces understanding and optimization within the Ruby ecosystem, simplifying previously convoluted documentation or C code comprehension. By integrating these routines, we foster greater transparency and boost potential for optimization.

00:23:09.960 As we draw conclusions from this discussion of Ruby's core libraries, we notice the possibilities of disentangling the core libraries into a Ruby-implemented structure, flanked by a smaller set of primitives. The right side would evolve into a refined version embodying several Ruby functionalities—Ruby practitioners will proceed with their implementations without concerns about fundamental changes.

00:23:41.880 Is it a good idea? Absolutely—there are immense benefits: more understandable code, greater shareability, less workload across various Ruby implementations, enhanced debuggability using standard tools, and more oxygen for optimization through methods like YJIT. Ultimately, we yield a more analyzable library and a path toward addressing and mitigating slow startup times while maintaining memory efficiency. Some unanswered questions remain along the way, but it's certainly worth exploring henceforth.

00:24:12.840 We've already cultivated a working core in TruffleRuby that we can experiment with; this exploration will become increasingly relevant as MRI and other implementations get more sophisticated through innovations like YJIT. This vision conveys promising possibilities for Ruby moving forward, enhancing performance, improving tooling, and adapting as the core library becomes more refined. I'd like to direct you to some additional resources for interested parties.

00:24:51.840 TruffleRuby is home to the core library implementation at the moment, so please check out the Ruby code there—you'll find it familiar, as we all know Ruby. For those interested in further developments, the official site for TruffleRuby can be visited at growlvm.org. My personal work is available at chrisseaton.com where I keep my findings about TruffleRuby. Moreover, a significant portion of the optimizations mentioned hinges on a technique called splitting. This sophisticated optimization will further bolster Ruby, so I recommend attending Benoit de los’s presentation about it at 3 PM in Room A.

00:25:29.520 Academic research is also taking place in conjunction with these ideas, including a recent publication by Sophie Kalb, who analyzes what core sites and calls look like in Ruby. Her work illustrates how pertinent optimizations in your Ruby logic can be, especially in optimizations connected to core libraries. I would encourage you to look up her paper for an insightful read. As we near the end of our talk, looking back at Rubinius, we recognize that it was an early Ruby implementation consisting of Ruby core libraries.

00:25:59.760 At one point, they wrote much of the VM in Ruby, even incorporating their garbage collector in Ruby. When we mention the mark-sweep technique as it pertains to garbage collection, this reflects practices found in Rubinius. Those interested can enhance their understanding by visiting rubycompilers.com, which chronicles the history of how Rubinius set forth core implementations.

00:26:31.080 Finally, I have a side project to share called the Ruby Bibliography. For anyone interested in diving into Ruby's research landscape, this page lists various Ruby-related studies. Thank you very much for your attention.

00:27:01.440 If there are any questions, I believe we have a couple of minutes.

00:27:04.200 Regarding the question about how we determine which primitives to keep and the trade-offs involved, it's a compelling question with many open issues to work through.

00:27:09.000 Currently, in TruffleRuby's approach, we default to implementing constructs in Ruby and only create primitive methods if we have a compelling reason. The rationale for particular primitives may fluctuate depending on their implementation style in different Ruby systems, such as MRI.

00:27:20.400 I do not have a singular resolution to this—it’s yet another unresolved question. However, we currently possess a functioning core library from which we can adapt and refine as needed.

00:27:29.760 Thank you again for your time. Feel free to come and find me with any further inquiries.