Methods of Memory Management in MRI

by Aaron Patterson

In Aaron Patterson's talk at RubyConf 2016 titled Methods of Memory Management in MRI, he explores various aspects of memory management specifically within Ruby's MRI (Matz's Ruby Interpreter) especially focusing on garbage collection (GC) algorithms. The presentation outlines important concepts surrounding how memory is allocated and freed in Ruby, which is crucial for developers aiming to enhance their applications' performance and manage memory effectively.

Key Points Discussed:
- Memory Layout: Patterson distinguishes between stack and heap memory, explaining that all Ruby objects are heap allocated. The heap is managed differently from the stack, which handles temporary function variables.
- Importance of Garbage Collection: Understanding GC is vital for dealing with apps in production, particularly concerning scaling and tuning performance. Knowledge of GC can reveal issues that impact application efficiency.
- GC Algorithms in MRI: Patterson elaborates on various collection algorithms:
- Mark and Sweep: This marks reachable objects and frees the unmarked, although it can slow down program execution.
- Generational Collection: Improves efficiency by categorizing objects by their age, allowing for faster GC cycles by skipping older objects assumed to be less frequently collected.
- Tri-color Abstraction: This incremental approach uses three colors (white, black, gray) to determine the status of objects in memory and allows for processing in stages, thereby increasing throughput.
- Allocation Algorithms: Ruby doesn’t call malloc every time an object gets allocated. Instead, it allocates chunks of memory (slabs) to create efficiency in memory allocation processes.
- Fragmentation Issues: Freeing objects can lead to fragmentation if not managed carefully, highlighting the necessity of efficient memory usage.
- Impact of GC on Performance: Tools like GC.stat and ObjectSpace APIs are discussed for monitoring GC performance and memory usage in Ruby applications.
- Challenges: Patterson notes that MRI's GC is non-parallel and lacks real-time guarantees, making understanding its processes crucial for optimization.

In conclusion, Patterson underscores the relevance of proper memory management and garbage collection in Ruby applications. Through thorough understanding and strategic management of garbage collection and memory allocation, developers can optimize the performance of their applications significantly, ensuring more efficient utilization of resources.

This talk provides essential insights into Ruby's memory management practices, offering practical tips for developers who wish to improve their understanding of GC algorithms and implement better memory management strategies in their code.

00:00:15.119 Good morning! Oh wow, I'm on a Saturday. I guess it's time; I'm ten seconds early according to this clock, but let's do this anyway. I have a lot of slides—251 according to Keynote—so this is the interactive portion of my presentation. Um, everyone, please unlock your phones, take out your phones, and take a look. I'm going to get this out of the way real quick here.

00:00:32.250 Okay, just 22 minutes of your time. Alright, everyone, I'm sorry. Okay, let's move on.

00:00:38.140 This is for me; this isn't for you, this is for it. Okay, thank you! Thank you for coming to my presentation about selfie sticks. There are two purposes to selfie sticks. The first important purpose is to help you identify people who like fun things and enjoy themselves. The second purpose of a selfie stick is to let you know who doesn’t like fun or fun activities.

00:00:51.610 So, I recommend that you all get a selfie stick. By the way, have any of you seen this photo around? Yeah, that's me! I've been sending airdrops to everybody here, and unfortunately, some people decline; it makes me sad. I like to do this often.

00:01:05.020 Actually, I was at an airport once, and there was a group of older people traveling together. They were coming home from vacation and said, 'Oh, let’s share all of our photos!' They were like, 'How are we going to do it?' and I heard one guy say, 'Oh, let’s just airdrop them to each other!' So, I sent them photos of my cat. One guy asked, 'What is this? Cat food?'

00:01:22.750 Now, if you're going to do this, make sure to name your phone something nondescript like mine, which is named "The iPhone". Let me show you a few of the photos that I have received. This is cute! We got another one here, and then I got this one. I love it when I hear about people asking, 'Who's doing all this?'

00:03:08.020 Yes, success! I've done it. Alright, so today we’re going to talk about methods of memory management in MRI or M Ruby. My name is Aaron Patterson, or Tenderlove, and that is my nerd code at the bottom, so if you want to send encrypted messages to me, you can. I noticed there was a cheerleading competition happening, which I think is amazing, but I like to think that it's probably a mockumentary filming, and I really want to see it.

00:03:43.360 This is what I look like on the Internet. I look different in person, so if you might recognize that icon more than me in person. I'm from Seattle, which is not Ohio. I want to share that my wife is from Japan, and we like to practice Ohio culture. Essentially, every morning, we say 'Ohio' to each other.

00:04:37.960 Yes! Haha, sorry, I only have 30 minutes of talk time. What's that? I work for a company called GitHub. It is a small startup out of California, but I don’t live in California; as I said, I live in Seattle. This company is the first legit company I've ever worked for, and I love it. But I'm not going to force push it on all of you!

00:05:02.070 So, the room is slowly turning against me! Again, I'm a GitHub certified engineer, which means I really enjoy bare metal. My name on there is Tenderlove, and it was weird starting at GitHub because everybody refers to everyone else by their nicknames. To give you a little background, I've been working remotely for about the past six years, so I don’t interact with many co-workers in real life. I went into the office, and everybody was calling me Tenderlove.

00:05:52.140 I politely asked, 'Please, you can call me Aaron. Tenderlove is fine too.' Actually, you know what? I'm going to tell a story, I don’t care; we're going to have to move very quickly. My parents are both engineers, so they’re not surprised that I sit at a computer and do my job all day. I tell them everything about myself, except for this one little thing: my name. They don’t know this name.

00:07:25.310 I was invited to speak at a conference in Salt Lake City, where I was born; my parents live there. I said, 'I'll speak at this conference, but only if you give me two free tickets for my parents.' Of course, the organizers said, 'Absolutely!' So I show up at the conference with my parents. We meet the conference organizer, who says, 'Okay, great! You're up soon, and we've reserved three seats at the front for you.'

00:08:44.340 So he takes us down to the front, and there are three signs. The first sign says 'Tender Love', the next sign says 'Tender Mom', and the last one says 'Tender Dad'. And I'm just like, 'No! Not now!' So I had to tell them, 'Look, this is just the name people know me by. It’s cool, don’t worry about it!' They didn’t know, and we've never talked about it since.

00:09:12.130 Anyway, I love cats. This is one of my cats, her name is Choo Choo. She isn't as famous as Gorby. This is Gorby, Gorbachev Puff Puff Thunderhorse! He is hiding here; he thinks he's hiding. He’s adorable and likes to sit on my desk, and I love the face she makes. It's the same face I make when I’m programming: just staring at a screen with no expression.

00:10:54.300 I have stickers of my cats, so if you would like a sticker, come say hello to me. I also have GitHub stickers; I think I ordered some from our online store. It said, 'Order some sticker packs, as many as you want!' I see there’s a drop down. It was kind of funny because the drop-down was numbers one through 200, so I thought if I ordered one, I’d get one pack of assorted stickers. Today, you wouldn’t order one random sticker.

00:11:57.130 So I thought, 'Well, at RubyConf, I’ll just order five,' which seemed fine. So, I ordered five, and then five more, but unfortunately, I forgot about the sprinkles. One of my co-workers, who is actually speaking right now as well, was nice enough to give me some stickers. I was recently in the news because I approved a pull request.

00:12:55.490 I'm also very much into keyboards; this is one of my keyboards. I love mechanical keyboards; if you want to talk about that, come talk to me. The interesting thing about this keyboard is that it’s backlit, but it’s backlit with ultraviolet LEDs, so I can get a tan on my hands. I don’t go outside very much, so I figure I should do that.

00:13:29.500 I want to talk a little bit about some new Ruby features, especially the ones that Matt was discussing in his keynote. He was talking about typing, and I want to talk a little bit about typing in Ruby.

00:13:41.130 So, let me show you soft typing. We have dynamic typing, which is a bit more complex, where you're moving your hand around the keyboard often, and then there’s static typing. In static typing, the way it works is that you don’t actually move your hand. The keyboard comes up to your hand! I’m really excited about these new features in Ruby 3.

00:14:45.240 So today we’re going to talk about GC. Let's get serious; I have 20 minutes to present 200 slides on garbage collection. First, I want to clarify two main areas when we talk about memory: the stack and the heap.

00:15:03.960 The stack is where temporary variables for each function are stored. When you call another function, you store some of that memory on the stack. When popping up from a function, the memory is released. The heap is unmanaged memory; you say, 'Hey, go allocate some memory,' which gets stored in a particular place; any function could access that memory.

00:16:02.240 In Ruby, everything is heap allocated. Inside the heap, there's one heap in terms of your machine, but inside that heap, we have what we call the Ruby heap where Ruby objects live. These Ruby objects can point to other places inside the machine’s memory. The Ruby heap is a subset of the entire heap.

00:17:10.230 You might be wondering, why learn about Ruby's GC? It's important, especially if you have apps in production; as it may encounter scaling or tuning issues. When this happens, it's crucial to understand garbage collection, as it may impact the performance of your application.

00:18:53.690 What I want you to take away from this is to learn the terms I'm going to use. If you already know GC terminology, pay attention to the algorithms; if not, just learn those basic terms. If you already know these algorithms, I'm going to share new information I’ve been working on in Ruby’s garbage collector.

00:19:21.130 So let's discuss GC algorithms in MRI. The collection side and the allocation side are important to understand. A garbage collector isn’t just responsible for reclaiming memory; it also handles allocations. First, we'll cover the collection algorithm, which seems counterintuitive since you typically want to allocate memory before collecting it.

00:20:53.410 The collection algorithms are first because they tend to be more complicated. In mark and sweep, garbage collection focuses on finding nodes no longer connected to the root and freeing them. The garbage collector starts at the root and marks reachable objects through the connections made.

00:21:54.490 After marking, we free the unmarked nodes. The mark-and-sweep process is relatively easy to implement, but it can be slow since it requires halting your program, which is not ideal.

00:22:26.620 One reason the mark and sweep process can be slow is because it has to walk through every single object each time, resulting in considerable time and resource consumption.

00:23:42.580 Generational collectors aim to improve efficiency by tracking the lifecycle of objects. Most objects tend to die young, and this can improve GC performance. For example, if we categorize objects by age, generational garbage collection lets older objects be treated differently than younger objects.

00:24:35.880 So, when using a generational strategy, if we identify B and D as older, we can skip them in subsequent garbage collection cycles. We only have to deal with newly created objects, helping to significantly reduce the workload of the garbage collector.

00:25:28.670 There’s a small issue, though. If new objects are created and referenced by older objects, we risk marking them as garbage even when they are still in use. To solve this, we can implement a remembered set that keeps track of these references.

00:26:31.400 The right barrier tracks these references and ensures objects aren't mistakenly collected. Sounds complicated, but the essence is simple: We identify which objects can be safely reclaimed and which should be retained.

00:27:55.090 Now, let’s discuss how to handle incremental garbage collection, which improves efficiency even further. This algorithm introduces the concept of three colors: white, black, and gray for objects. White denotes objects that will be collected, black signifies those with no references to white objects, and gray indicates objects that reference the root but require further examination.

00:28:47.580 In this scheme, we can halt at any step and process the collection incrementally, leading to reduced halting time and increased throughput. The main advantage of the tricolor algorithm is it allows processing in stages, thereby enhancing performance.

00:29:50.510 However, the algorithm still needs a right barrier to track references to prevent mistakes during garbage collection. This ensures we keep track of objects and make sure we do not free memory needed by the application.

00:30:45.810 Let’s talk about what MRI's garbage collector does not do. Our GC is not parallel, meaning it doesn’t execute in parallel with your program. It runs concurrently through its incremental steps, but it doesn’t offer real-time guarantees on GC execution time.

00:31:56.750 It is also not compaction, meaning that it does not reorganize objects moving them in memory as needed. This is crucial during garbage collection, as fragmentation can lead to inefficient memory use and allocation.

00:32:57.490 Now, talking about allocation algorithms in Ruby, we do not perform a malloc every time we allocate an object. Instead, Ruby allocates a chunk of memory, referred to as a slab, and subdivides it into individual Ruby objects, thus increasing efficiency. This reduces CPU time consumed by frequent malloc calls.

00:34:33.300 Each slab contains a linked list of free slots, where each slot houses a Ruby object. As you allocate a new object, the pointer moves along the linked list to find an open slot, which we call bump pointer allocation.

00:35:46.780 In cases where a page fills up, Ruby’s GC allocates more pages to free up memory and ensure adequate availability for new object allocations. The strategy behind this allocation scheme is efficiency—allocating larger chunks reduces overhead.

00:36:55.730 What’s interesting is that not every object in Ruby necessitates an allocation. For example, integers and floats utilize a tagging mechanism to represent values without requiring allocations.

00:37:59.440 This clever approach allows values to be represented efficiently in the system memory, which aids in reducing the frequency of allocations and enhancing overall performance.

00:39:00.500 Nevertheless, Ruby objects create issues during reclamation. If we free some objects while still holding on to the empty pages, we're left with fragmentation in memory, leading to inefficient usage.

00:40:05.390 To improve space efficiency and reduce GC time, grouping older objects may help. If we can cleverly predict which objects are likely to age, we can optimize the space they consume.

00:41:15.300 I’ve been thinking about methods for dealing with this at GitHub and how to optimize memory management. The goal is to decrease allocation times by grouping similar object types together.

00:42:03.500 Finally, you can view GC data using the GC.stat command or use profiling tools to review garbage collection impacts more comprehensively. This provides insight on performance, memory issues, and allocation rates.

00:42:55.240 You can further inspect objects using ObjectSpace APIs in Ruby, which can provide detailed information on individual objects and their memory footprint.

00:43:33.550 At work, we tweak certain environment variables to optimize our Ruby applications, particularly focusing on maintaining a larger heap size to minimize interruptions during execution.

00:44:58.330 By understanding these garbage collection processes and making intelligent choices about memory management, we can significantly enhance our applications’ performance. Thank you very much!