Implementing Object Shapes in CRuby

Talks

Jemma Issroff

Implementing Object Shapes in CRuby

by Jemma Issroff

The video titled Implementing Object Shapes in CRuby features Jemma Issroff, who discusses the implementation of object shapes in CRuby as part of the upcoming Ruby 3.2 release. Object shapes are an under-the-hood feature designed to enhance performance by optimizing instance variable lookups. In her talk, Jemma answers three key questions related to object shapes:

What are Object Shapes?

Object shapes refer to a structure that represents the properties of Ruby objects, allowing for efficient instance variable lookups and cache hits. Unlike static types in languages like Java, Ruby's dynamic nature means object properties can change during execution, necessitating a mechanism like object shapes to encapsulate these properties.

For example, different instances of similar classes can share the same shape based on their instance variables, improving memory and performance efficiency.
How are Object Shapes Implemented?

The implementation involves assigning unique identifiers to shapes and creating a structure that allows Ruby to track properties efficiently. When an object is created, its shape starts at a root and transitions through identifiers based on the instance variables set. This creates a hierarchy or tree of shapes that enhances caching.

On different architectures, 64-bit systems can support around 4.2 billion shapes, while 32-bit systems can accommodate approximately 65,000.
Benefits of Object Shapes

The primary benefits include:
- Increased cache hits due to the elimination of class name dependencies, leading to more efficient indexing of instance variables.
- Decreased code complexity: Object shapes reduce the frequency of frozen checks when setting instance variables.
- Enhanced memory efficiency during allocation by directly accessing shape trees, avoiding the need for pre-populated memory.
- Improved performance with JIT compilers through reduced instructions needed for variable access.
  Jemma cites performance improvements seen in both Rails benchmarks and micro-benchmarks, stating that object shapes can provide over twice the throughput compared to previous methods.

In conclusion, the implementation of object shapes significantly enhances Ruby's performance by optimizing how properties are represented and accessed, leading to faster execution times and reduced overhead in code complexity. Jemma encourages connections in the Ruby community and highlights events fostering inclusivity for women and non-binary individuals in the tech space.

00:00:06.540 Many of us might be familiar with the Ruby release announcements that look something like this: we're used to seeing them every year when a new version of Ruby is released. As Matt discussed in his keynote this morning, this year's version will be Ruby 3.2. We can expect another announcement similar to this when it is released, typically around Christmas, which will naturally include some updated features. Pulling together these slides has been enlightening, as I realized that Ruby is like our best friend, consistently providing us with new features. With each Ruby version, we receive several new features that enhance our development experience. Some features are very visible to end users, while others focus on performance improvements, security updates, and similar enhancements that might not be immediately noticeable. Each year, as we release a new Ruby version, we see a mix of these visible features, such as pattern matching or new reactors, and the behind-the-scenes improvements that help make our experience better. Today, we will be discussing a particular implementation that falls into this latter category. I have been collaborating with Aaron Patterson over the past few months on implementing a technique called object shapes in CRuby, which will be included in Ruby 3.2. Unlike feature updates that are easily visible, this implementation is more of an under-the-hood improvement. However, we will delve into what this means, its performance impacts, and what exactly object shapes are.

00:01:10.560 Hello, I’m Jemma. If we haven’t met yet, I would love to connect! I work at Shopify on the Ruby and Rails infrastructure team, where I have been able to work on this exciting project. During today's talk, I intend to answer three main questions: First, we will explore what object shapes are. Next, we will discuss how they are implemented, and finally, we will examine the benefits of object shapes and the reasons behind their implementation. To start, what are object shapes? When we hear the term ‘shapes,’ we might picture geometric forms like diamonds, triangles, or ovals, but that's not what we mean in this context. In fact, when we talk about object shapes, we refer specifically to Ruby objects—those we frequently interact with using object.new, for example. The term ‘shapes’ is meant to denote a concept and isn’t related to literal shapes. This technique has been around since the days of Smalltalk and represents the properties of Ruby objects. Each Ruby object possesses a shape that signifies its properties, such as the frozen status indicating whether the object is frozen and its instance variables.

00:02:11.480 When we discuss the need for this representation of an object's properties, we must remember that Ruby is a dynamic language. Unlike languages such as Java, where we know all the information about an object upfront, Ruby allows properties to change during the execution of our applications. For instance, we might add more instance variables or freeze an object at runtime, which alters its properties and shape. The idea is that we can leverage the concept of shapes to encapsulate all properties, which perfectly illustrates why this is significant. Now, let’s look at a practical example. Consider a class named ‘Post’ which has a title and an author. When we create an instance of this class, we can analyze its shape.

00:03:04.660 In this instance, the first aspect of the shape of ‘First Post’ includes its instance variables, namely `title` and `author`. The shape holds these instance variables, thus providing an organized representation of the object. The shape also includes a frozen status. If we were to freeze ‘First Post’, its frozen status would be marked as true within its shape. Additionally, shapes have unique identifiers or IDs that distinguish them; for instance, we could arbitrarily assign the ID of 6 to this shape. Let’s explore another example with a class called ‘User’. Similarly, when we create an instance, we note that its instance variables are `name` and `login`. For this class, we could assign it an ID of 31. If we create a separate class called ‘Admin’ with similar instance variables, the object created from this class could have the same instance variables yet be distinct from ‘User’. However, what we see here is that both ‘User’ and ‘Admin’ can share the same shape despite being different class instances.

00:03:43.120 This sharing of shapes is a critical concept. The properties attached to an object generally inform its shape, meaning that these different instances can have identical properties leading to them sharing the same shape. When new properties are added to a shape, everything starts from what we call the ‘root shape’, which has an ID of zero and no instance variables—it’s practically an empty shape. When we first initialize ‘First Post’ with the method `post.new`, we start with the root shape. As we set the title by transitioning through an edge named `title`, we form a new shape that contains only the `title` instance variable and gives it the next ID. If we then set the author, we transition again through the instance variable `author` to create a new shape that now holds both properties: `title` and `author`, along with the next ID. However, if we split the implementation of the post class into separate methods, say initiating the title in one method while setting the author in another, ‘First Post’ would hold a shape with ID 1 (only containing the `title`). If we were to add another instance called ‘Second Post’ and set the author afterward, it would have a different shape (with ID 2). Thus, it’s important to acknowledge how distinct instances of the same class can have different shapes based on how we manipulate the properties.

00:05:07.080 Now, consider another class known as ‘Image’, which also has a title. However, unlike ‘Post’, it features an image URL instead of an author. When we call `image.new`, we start again with the root shape, transition through the edge of `name` to arrive at shape ID 1. Rather than proceeding with the author transition, we now set the image URL which transitions us to a new shape with instance variables of `title` and `image URL`, giving it the next ID. All of these shapes collectively form a tree structure. To clarify things, when we previously mentioned the concept of shapes transitioning with new properties, we can visualize this transition by looking at our shape tree. Let’s take an anonymous post as an illustrative example—one that only contains a title and no author. If we have an instance of this anonymous post, we begin at shape ID 0, then transition through the title, leading us to shape ID 1.

00:05:50.000 If we then decide to freeze our anonymous post at this point, we wouldn’t take the existing paths; instead, we would make a new transition through a frozen edge to a new shape that still retains only the `title` instance variable but classifies the shape as frozen. This gives us an overview of what object shapes are and how they operate. Now, let’s dive into the implementation details of object shapes, exploring the behind-the-scenes work that makes this possible.

00:06:41.040 To identify the shapes, we’ll look at the structure we’ve created. Each shape is assigned a unique identifier or ID. This ID indicates the precise properties that shape embodies. We have the capacity for numerous shapes across various systems, running on 64-bit and 32-bit machines. On 64-bit machines, we utilize 32 bits for shape IDs, providing around 4.2 billion possible shapes. Conversely, on 32-bit machines, we use only 16 bits, which allows for around 65,000 unique shapes. To put this into context, within the Rails benchmarking suite, we typically witness about 10,000 shapes, while my tests on Shopify’s extensive monolith revealed roughly 40,000 shapes. Both these figures comfortably fall within the threshold of 4.2 billion shapes available on 64-bit systems.

00:07:44.780 Next, let’s address what information is encoded within a shape. Ruby, as many know, is written in C, and it uses an RB shape struct, which encompasses various properties. For instance, an ID table functions as a mapping mechanism between keys and associated values, defining which edges exist on a particular shape. Each edge is characterized by the instance variable count, detailing how many instance variables exist on that shape. For example, take shape ID 2 and shape ID 6—both have a count of two instance variables. Shapes also have types that categorize them into four distinct types. First, there’s the root shape, which we discussed earlier. Next are instance variable shapes forming the bulk of our structure, having been generated through instance variable transitions. Following that are frozen shapes; these shapes are leaf nodes because they indicate that no further instance variables can be added. To create a frozen shape, we must make a frozen transition. Lastly, we have what is known as an undefined shape. This shape appears when an instance variable has been defined and then later removed, transitioning to this undefined state.

00:08:34.840 Now, let’s look at how object shapes were implemented and the benefits of that implementation. The main question remains: why did we decide to incorporate object shapes? Let’s analyze several advantages that object shapes introduce. First, they bring about an increase in cache hits—something beneficial to all programmers. For object shapes, we revamped how caching works with the reads and writes of instance variables. This means that how we operate with instance variables in Ruby 3.2 has shifted compared to prior versions. To understand the modifications, it's crucial to revisit how instance variables work in Ruby 3.1. With the instance of our `Post` class, each instance has its own array to store values through an index system. When we set an instance variable like `title`, Ruby checks the variable name’s existence in this indexed map—if it doesn’t exist, it’ll add it at the next available index, which is assigned a value within the instance variable array. However, this process requires repeated hash lookups, which can be mildly expensive to compute. Therefore, a better solution is clear: caching can help minimize such repeated checks. Previously, the class name served as the key, which led to cache misses whenever we had inherited classes, despite maintaining the same initialization method.

00:10:10.360 With the introduction of object shapes, the dependency on class names for caching was eliminated. Instead, we now leverage the shape tree structure. The shape ID and instance variable count allow us to calculate the index directly without needing to access a class or maintain a map—making this process seamless and efficient. This means that cache hits will now be more frequent, especially when dealing with inherited classes. For example, if we set `title`, the shape ID allows us to find and cache the index determine based on its instance variable—hence increasing efficiency and access speed. As we transition to setting the `author`, we can apply the same method using the shape ID, causing further optimization and allowing us to avoid the cache misses we previously faced. The cache now keys off the shape ID rather than the class name, enhancing performance throughout.

00:11:32.640 The second significant benefit of object shapes lies in decreased code complexity. This doesn’t just occur through performance enhancements, but we also see a reduction in necessary code checks, particularly those regarding frozen checks in instance variable setting. In Ruby 3.1, to set an instance variable, several arguments needed to be analyzed to confirm the state of the instance before completing the action. Notably, we had to assess if the object was frozen, which added a layer of complexity. However, with object shapes, this frozen check can be made less frequently. During our checks, when we run into a cache hit, we don’t need to worry about the shape being frozen since that transition has already been established in the cached state. Moreover, our implementation ensures that as long as it gets cached, we can safely assume it isn't frozen. Thus, we significantly streamline the setting of instance variables to a swift process. To illustrate how we further reduced code complexity with object shapes, let’s explore how we deal with memory allocation. Typically, when we call `object.new` in Ruby, memory is allocated for our instance variables. Previously, Ruby populated this memory with placeholders set to ‘undead,’ which requires additional checks for whether each instance variable is defined or undefined. This approach sounds reasonable, but it comes with performance and complexity costs whenever we allocate new objects.

00:12:56.400 With the advent of object shapes, we’ve mitigated these costs drastically. No longer do we need to pre-populate memory with ‘undead’ values during allocation since the instance variable checks transition through the shape trees. When we begin looking for an instance variable’s index, we can navigate through the shape tree to find it and this way avoid unnecessary overhead costs. Essentially, we've reached a revolutionary point where we populate our memory allocation only with what we need, improving performance and drastically reducing complexity. Lastly, the benefits extend to Just-In-Time (JIT) compilers. JIT compilation is critical for enhancing performance in Ruby applications. By minimizing the number of instructions that need to process when accessing instance variables, object shapes streamline execution. When we analyze `get` or `set` operations with JIT compilation with the old methods, the operations are verbose and include numerous comparisons. With object shapes, we drastically reduce this complexity—fewer instructions to execute means a more reliable and faster performance feed.

00:14:38.010 We also noticed substantial performance gains while evaluating both our Rails benchmarking and micro-benchmarks. In Rails bench, we measured average requests per second, and upon comparison, we found shapes provided a higher throughput. The results indicated shapes not only improved performance but also offered more precision in execution. Changes were generally observed when testing consistency across standard instances in Ruby with the fares of getting or setting instance variables. Some tests illustrated shapes consistently outperforming our previous iterations by over two times in certain benchmarks. In conclusion, we’ve effectively answered three key questions: what are object shapes? How are they implemented? And finally, why did we choose to implement them? Thank you all for your attention and thanks to Aaron Patterson for collaborating on this significant work.

00:15:38.120 I invite you to connect with me on social media, as I will post details about further developments regarding shapes implementation. Additionally, I want to highlight a new community-focused event called RubyConf Mini, set to launch in a month on the East Coast of the U.S. Please explore if you are interested! Finally, I manage a women and non-binary Ruby community called WNB.rb. If you identify as a woman or a non-binary person, I encourage you to join. We will be meeting right after today’s session and heading to a nearby coffee shop to socialize and connect. Thank you once again for your support and attention!

EuRuKo 2022