Reducing Enumerable - An Illustrated Adventure

by Brandon Weaver

The video titled "Reducing Enumerable - An Illustrated Adventure" features Brandon Weaver and was presented at RubyConf 2018. This engaging presentation follows the whimsical journey of Red, a reducing lemur, as he explores the capabilities of the reduce method in Ruby, especially in the context of newer methods introduced in Ruby 2.4, such as sum.

Key Points Discussed:
- Introduction of Reduce:

- Red loves to use reduce to sum values.
- reduce starts with an accumulator and processes a list by combining values using a designated method.

- Ruby's sum Method:

- The introduction of sum in Ruby 2.4 raises questions about the necessity of reduce.
- Red learns that reduce can do more than just summing—it can apply different operations and work with structures beyond simple numbers.

- Adventures with Masters of Enumerable:

- Red embarks on a quest to learn from three masters:
- Master of Map:

- map transforms the original collection and retrieves a new list based on a function applied to each element. Red learns to implement mapping using reduce.

- Master of Select:

- select filters elements based on a condition. Red discovers how to apply select operations utilizing reduce to build a new array of elements meeting the criteria.

- Master of Find:

- find retrieves the first element meeting the condition. Red learns to implement find using fold-like behavior of reduce by leveraging early termination when a match is found.

- Advanced Usage of Reduce:

- Red’s journey concludes by discussing how reduce can handle more complex operations, like counting occurrences using a hash (tally by), and how understanding the functional programming aspects can enhance coding practices.

- Conclusion and Reflection:

- The master reminds Red that reduce is not obsolete; it serves its own purpose where other more direct methods may not suffice.

- The real power lies in making informed choices about which methods to use based on the problem context rather than defaulting to one approach.

- Final Thoughts:

- The journey embodies the idea that there are always new avenues to explore in programming, and creativity in using methods like reduce can lead to innovative solutions and better understanding of functional programming principles.

The session closes with a sense of humor and anticipation for future explorations in Ruby's capabilities, showcasing the importance of community engagement and the potential for continuous learning.

00:00:15.560 Well, it looks like we're about ready to go. Now, I cannot promise cartoon raccoons, but I can promise cartoon lemurs, and I swear they are just as good and just as fun. So just because it's a magic talk, I enjoy a little bit of theatricality. We brought along a magic book. You might wonder what the magic book is for—well, you see, every time I turn one of these slides, something magical happens.

00:00:30.210 So, who exactly am I? My name is Brandon Weaver. I am an artist who ended up becoming a programmer. How that happened was through a series of unfortunate accidents that started with getting into web design. A little bit of HTML and CSS made me more marketable, along with a little bit of JavaScript. I had a little bit of animation here and there—sparkle, a little of this and that. That seemed insane, because next thing I knew, I was diving into systems engineering. Currently, I'm doing that over here at Square, where I work as an infrastructure services engineer on the platforming team.

00:00:52.649 Amusingly, I am working on the same code that Jack Danger happened to mention in his talk. That's a little bit of fun! As far as social networks go, you can find me on Twitter, GitHub, Medium, IRC—yes, people still use IRC, I know right—and Ruby.social, which is one of the new Mastodon instances. You will notice that I left tags to my Twitter on the bottom of a lot of these slides, so you will be able to find it later. I'll go back over this at the end of the talk.

00:01:16.229 So, shall we get started? This is Red. You see, Red is a magical lemur, and he loves to reduce things. He absolutely adores it! He can take stacks of numbers, oh, a day long, and he just loves reducing them into sums. To him, it is the most amazing thing ever. So, let's take a quick look into how reduce works.

00:01:58.229 Reduce takes a list of numbers. Let's say 1, 2, and 3 for this example. It's taking that and reducing it into the number 6. What this looks like is a bit hard to decipher, especially if you don't fully understand what it is doing. I know I certainly didn't for many years, and I probably still don't in some cases, but we will ignore that part. So let's take a look at what it actually does.

00:02:36.250 Reduce starts with an accumulator, which you'll see highlighted in red (and underlined for the color blind or otherwise). An accumulator often starts as an empty value; if you add anything to 0, you will get back that number (1 + 0 = 1, 0 + 1 = 1). If we have an empty list, we can return something as a default. Next, we have the actual list, which you'll see noted here in what you might call a Cayenne or teal. I prefer to refer to it as teal. This will be present throughout this talk as 'v'. Finally, the last and most important aspect is how do we join the accumulator and the value together, which we'll denote in green as our method of combining things.

00:03:17.620 Basically, number plus a number gives you back a number, so it ends up looking something like this: 0 plus the list of 1, 2, and 3—or in more common parlance, something like that. Let's take a look at what this is doing step by step. We start with an accumulator of 0, then we add the first value, which is 1, resulting in a new accumulator of 1. Every step of reduce the accumulator carries over and becomes the accumulator for the next step.

00:04:00.370 So if we go to the next step, our accumulator is now 1, and we add that to the next value in our list, which is 2. This gives us back the value 3. Therefore, on the last round, we have 3 plus our last value, which is 3. Once we run out of values, reduce knows it is done and can return the accumulator, which is 6. So, by now, we can see that we have a list of many things reduced to one thing, using an initial often empty accumulator value, such as 0, along with a way to join two values together.

00:04:54.710 Of course, this makes us masters of functional programming, and we may feel that we have mastered everything there is to know. There is no point in really learning anything else. However, the problem arises because Ruby 2.4 came out with an interesting little function that I know some of you are thinking about as I did. That function would be 'sum'. So now my talk seems kind of pointless.

00:05:13.910 Well, this raises the question: Is reduce unnecessary? I wouldn't say so. This is not the end of the story. Red decides to ask his master if 'sum' kills reduce, and then something magical happens. Red receives a letter. In that letter, he finds that his master replied and said, 'Come visit if you wish to learn.' So it looks like we are going on a little adventure to find the true powers of reduce from Red's master, Scarlet.

00:05:50.180 Thus, Red journeyed over the rolling hills, through the grassy plains, until he saw a castle upon the horizon. That castle was nestled between mountains, where the clouds reached the sky, and therein he found the great and wise master of reduce, Master Scarlet. Red told his master what he had learned about summing lists together by using addition and all the amazing things he discovered. Yet, he still had that nagging question about reduce.

00:06:12.560 Red simply had to ask, 'Does something make reduce useless?' Master Scarlet paused for a moment, then responded, 'Perhaps you can do more than just sum with reduce. What if we used subtraction, multiplication, division? What if we didn't use numbers at all? Perhaps instead, we have an empty array and we push to add elements to it.' Master Scarlet then said, 'You will find three masters in the land of Enumerable. Go out and learn from them about their functions, and see if you can figure out how to do the same with reduce.' With that, Scarlet left Red with much to think about and a map to guide him.

00:06:45.199 Now off he went to the corners of the map to find the masters and learn how their functions work and how he might use them with reduce. The first master he was to visit was the Master of Mapping. Red journeyed through the forests of Transformation, deep into the groves where mushrooms bloomed and creeks ran sparkling blue. He discovered a path leading to a house, and outside the house, he met the first master he was to see: Master Chartreuse of Map.

00:07:13.539 'Welcome, Red! I've heard so much about you and your journey from Master Scarlet,' said Chartreuse. 'Wise master, can you tell me how map works so that I may implement it with reduce?' Let me show you young one, the ways of mapping. To map is to apply a function to every element of a list and get back a new list. Let's examine our list of 1, 2, and 3, where we wish to multiply every element by 2. Each element is doubled: 1 multiplied by 2 is 2, 2 by 2 is 4, and 3 by 2 is 6, giving us back an entirely new list of 2, 4, and 6.

00:08:08.130 Red began to wonder how he could apply this to reduce. His new list was empty, and his joining function was to push. So how might we use this? You see, what we start with is pushing a new element to the list for every step we take. But before we do that, we call a function to transform that element into what we want it to be. Taking a look at this all together, it could look something like this, but that’s a lot of code and personally, I find it very unsettling. Yes, I agree it is.

00:09:11.240 So let's dig into it and clarify what exactly is happening. We have our old familiar list, which is passed as an argument. For each value, it comes across as 'v', and then we have our accumulator, which is now an empty array denoted as 'a'. Our joining function this time is 'push'. If you use 'push' on an array, it means you add a new element, but before adding it, we must call the function that we were given with the value to transform it before we do anything further.

00:10:09.120 The idea here is that we're pushing onto our accumulator the result of calling a function on a value. This can be a bit of a mouthful, I do agree, but let’s take a closer look at what that looks like. So, using this structure, instead of enumerable methods, we are using 'v' within the function, which is a block. Ruby remains on the outside, and still, we get back the results, which are 2, 4, and 6.

00:11:02.160 To map is to apply a function to every element of a list to create a new list. To map with reduce is to apply a function before appending an element to the list in the first place. With the lessons learned from mapping, Red decided to visit the next master, Select, and see what there was to learn. But on his way, he found a peculiar tree, and when Red looked up, it seemed as though things were going backward—he had no idea which way was up anymore. On the ground, he found a hat.

00:11:51.400 Now, you see, in the lemur world, hats meant there was a master somewhere nearby. Surely only masters wore hats. So Red picked up the hat and wondered aloud, 'Is there a magician? Is he here?' Then the tree reached down a branch and said, 'Hello there! Would you be so kind to give me my hat?' 'Are you a master?' asked Red. 'Well, yes I am. I am Master Branch.' You see, I get blamed for knocking it off again, but it was probably the cat trying to shake it off.

00:12:43.010 At this point, Red noticed a hastily scrawled sign saying 'Beware of the tree.' Now, with the lessons learned from map, it was time to actually go to the Master of Select and hopefully not encounter any further oddities. Unfortunately for Red, the Master of Select proved to be nothing but oddities. He journeyed through towns and villages looking for the Master of Select, and as he got closer, things kept getting stranger and stranger until he eventually found himself surrounded by absurdities.

00:13:27.210 At the end, he discovered a house, and within that house was the Master of Select. When Red met Indigo, the Master of Select, he found him to be quite eccentric. Select had a collection of various Ruby artifacts and toys. Red noticed something peculiar: both the tree and Indigo were accompanied by a cat. He asked, 'Is that a Cheshire cat?' Indigo responded, 'No, no, no, we're not in Wonderland. Here, we call it a Sure Cat.' Red, feeling a bit disturbed, decided to continue with his original plan and ask how to find anything in this collection.

00:14:06.310 Indigo replied, 'I can find all the rubies in this room quite easily. I just have to zoom around, and there we are, done!' Curious, Red decided to ask the wise master, 'How may I implement select using reduce?' Indigo chuckled and replied, 'Let me show you a magic trick. I take a number and a function, and if that function returns true, poof, it's gone—never to be seen again. Let's take a quick look: select uses a function to decide which elements should go into a new list.'

00:15:02.990 We have our favorite old list: 1, 2, and 3. If we want to get even elements, our function checks each number: for 1, it’s not even, so we discard it, while for 2, it is even, so we keep it. Finally, for 3, it’s not even, so we discard that as well. Thus, we receive simply a new list with just 2. So, Red pondered how he might implement select using reduce.

00:15:45.750 Like map, we use a function for select, but instead, we apply this function whenever we decide to push an element onto a list. We only do this if the function returns true. This yields something like the following implementation, but you might notice something odd—we still have to return the accumulator here. That's because, if you use an 'if' statement, reduce doesn't like nil, and nobody really likes nil. Some might suggest using 'each_with_object,' but that doesn’t fit on the slide, so here we are.

00:17:03.150 The essence of our operation is that we’re pushing a value onto the list if the function returns true when provided the element. In any case, we must always return the accumulator. This means we have a function that looks like this: essentially very similar to our mapping example from earlier, where we have a list of 1, 2, and 3, and we're selecting even numbers. It gives us back only the number 2.

00:17:43.560 Next, Red set his sights on his final destination: finding. But, as he surveyed the route on his map, he realized it was quite excessive to walk all that way—for instance, why go through all that trouble when a train could take him? Thus, a train was constructed, and it was dubbed the Ruby on Rails. However, he had no idea why it was named that, but in any case, it was comfortable.

00:18:27.300 So, Red was on his way to the Master of Find. As he made his journey, he eventually came upon the city of Find, where the master had crafted amazing things with all the research she conducted. He noticed a conspicuously placed teapot since he had an odd sense of humor. He left the rails and entered the town, where he found her—Caffeine—with a crown on her head.

00:19:07.900 He discovered a vast library stretching as far as he could see. Inside, in the deepest alcove, he found his next master: Violet, the Master of Find. She was diligently looking for a very particular book. 'Ah! So you must be Red! I've heard so much about you—come in, come in! I’m just looking for a particular book at the moment.' So, as she searched through all the books in the library to find just one, Red couldn’t help but think: how silly would it be to keep looking for a book after I've already found it? That's precisely the role of the find function. Could you demonstrate how it works, wise master?

00:19:48.600 With pleasure! Find works similarly to select, but once we discover what we're looking for, we stop looking. If nothing is found, well, we found nothing. There are ways to handle defaults here, but we won't worry about those today. So, Violet began explaining find: it uses a function much like select. When we have a list of 1, 2, and 3, and we want to find the first even number, we check each element. If it’s not even, we keep looking, and if it is, we stop looking. Therefore, we don't even check for 3, and instead, we stop the iteration.

00:20:47.540 Essentially, that gives us back the value of 2. Red had to think about this because this idea was quite foreign to him when it came to using reduce. You see, you're returning a single value here instead of an array as you would normally expect. So how exactly does this work?

00:21:34.640 We don't even care about joining or accumulating; all we care about is whether the function returns true. So, by this logic, we break with a value if a function returns true, much like select. That's right—'break' is a keyword that allows you to break out of a loop and return a value. But even the example here looks nothing like what we have seen so far using reduce. It appears quite mad!

00:22:10.230 You see, we’re returning nil with this setup. We don’t even care about the accumulator. You mean to tell me we can practically use reduce in the capacity of an 'each' loop? Yes, although this may not be a conventional practice, I might get dinged in code reviews for it, but let’s ignore the possibility for now. We have a way of breaking out of our function just by providing a value, but that only happens when our function call returns true.

00:23:17.860 This leads us to a function that looks like this: If we never decide to take a break, how could we expect to receive any results? I suppose in the trend of things, that also gives another metaphor for software development, doesn’t it? So, like our previous example, find takes in a list and a function. In this case, we return the value 2 with our unpacking to demonstrate the nuances of this remarkable function.

00:24:20.849 To summarize, find just utilizes a function to locate a single element from a list, and once it finds it, it stops looking. To find with reduce merely reduces until it finds that match—a true mouthful. Now, with his journey done, Red decided it was time to return to his master. On his return trip, he encountered another set of lemurs—a rather peculiar trio: Cerulean, Saffron, and Vermilion.

00:25:07.890 Cerulean was clapping on his tambourines while behind him two other lemurs, friends of Vermilion, sang jubilantly. Naturally, Red was curious and enquired, 'What's going on with those other two lemurs back there?' Ah, that would be a chaperone and Vermilion. You see, they make every trip go a bit faster, but the problem is they never listen unless you happen to play the fiddle. I have yet to figure that out. Yet at least they do listen to that tune.

00:26:02.060 'Why the journey to see Master Scarlet?' Red inquired. 'That’s because I’m on a quest to establish a new school of enumerable tally by. We've heard from the Council of 2.6 that it might be forthcoming, and Grandmaster Nobu may very well have ordained the code already. Would you be willing to show me what count by does?' Tally by, you see, looks like map in that it leverages a function to affect a value but differs in that it utilizes a return value as a key to keep a tally or count of something—true or false, or maybe the first letter of a word or something else entirely.

00:26:47.860 Then Cerulean began to explain the intricacies of tally by. Let's examine this with the elements 1, 2, and 3, and we will consider whether or not they're even. The result shows that one is not even, leading us to one count of false; for two, it is even, providing one count of true; for three, it isn't, giving us a total of two counts of false and one count of true. Therefore, we are returned a hash of true with a count of one and false with a count of two, but that doesn't look like an array; I don't need something new altogether.

00:27:43.750 So instead of reducing into an array, perhaps we can reduce into a hash. For those unacquainted, that zero there indicates that any key unknown will have a default value of zero, allowing us to add to it indefinitely. To align with the methodology established previously, we might end up with something akin to this; yikes—going back; that truly terrifies me, and I would rather not see that code again.

00:28:36.450 We are playing it a little fast and loose, as Ruby is not technically a fully functional language, so we can escape doing things we ought not to do. Divulging this, we are operating a function on a value to create a key, and as we construct our accumulator hash, we’re adding one to that key's count. So let us inspect our tally by function once more: as always, this is a complex presentation, so let’s break it down.

00:29:20.300 Reflecting on our previous examples, we first find the key, akin to mapping. We are applying a function to a value to derive this novel value, and through that, we are modifying our hash by adding one to the counts of that key. This can be represented as something like 'word.first', or in the case of Ruby, would look something like 'word[0]'. Overall, our function might resemble the following; suffice to say this is still very dense.

00:30:37.580 With this method established, we start with no counts initially, and then we proceed with the first value, one. This is not even, resulting in false, thereby adding to our counts of false with a return value of one. Consequently, upon the next iteration, we commence with the tally of false at one with the value of 2, which is true, thereby increasing the count for true to one. We then find ourselves with a count of one for false and one for true, and if our last element happens to be false upon check, we receive an additional count of false, resulting in a final outcome of two for false and one for true. Thus, tally by is fundamentally counting the elements of a list after applying a function to them.

00:31:39.499 Essentially, tally by will reduce using a hash to maintain the counts of values the function generates. After he said farewell and wished him luck on the Council of 2.6 meeting, Red found himself back where it all began. He recounted to Scarlet all he had learned and the time spent in the realm of Enumerable and all the adventures he had. 'Look at all the new functions I've learned! But I still have to ask, after learning only these related functions that already exist, should I now use reduce for everything? Is reduce itself unnecessary?' So many fascinating lessons remain, Red, but this is just the beginning of a grand journey for you—a grand journey of knowledge.

00:32:51.350 Consider for a moment: would you use an axe to trim a bonsai tree? Certainly not, that would be quite absurd! Would you use hedge clippers to cut a mighty redwood? No, so does that mean I can't use reduced quietly? The challenge, Red, lies in distinguishing between the two and understanding when to wisely approach each method.

00:34:02.119 Reduce has its own applications when simple functions are insufficient, but that doesn't answer your question completely. We’ll need to delve deeper into the true powers of reduce that go beyond what you already know—there's real power here, young one! But it's crucial we learn how to utilize this power wisely. Concepts such as composition, carrying, closures, transducers, category theories—so many more intriguing ideas are out there to explore. Those are the precise reasons why learning and using reduce is so rewarding.

00:34:20.619 Imagine combining a function that reduces a list while integrating actions for both selecting and mapping—if a function returns any value except false or nil, that result is pushed onto the new array after transforming it. This is precisely what transducers do, but Red, that's a story for another time.

00:34:42.370 What you’ve experienced encapsulates just the initial steps on your journey of knowledge. One day, you will walk that other road! Yet today, I regret to inform you that our time is drawing to a close. Soon, you will be returning home, but I eagerly look forward to seeing everything you've accomplished and all you've learned.

00:35:15.970 As Red journeyed back home, he had much to contemplate. Realizing how little he understood about reduce and all the time he had spent learning already—and how much more there was to discover in the world—was truly a beautiful thing. Even after all that time spent adventuring, there so much more to learn, so many more beautiful things to find, friends to meet, and people to greet. Friends to laugh with, friends to cry with—everything beyond what could be imagined, and to him, that was the most valuable part of this entire journey.

00:36:09.130 Perhaps this is the end of Red's story, or perhaps it is merely the beginning of a new adventure altogether. To wrap things up, all the illustrations in this talk were genuinely my own work—about 50 illustrations, spanning 200 hours, and alright around 140 slides. Can you believe that?

00:36:38.379 You will find tested examples that will provide you additional information in the repo, which I'll pin and tweet out later. And again—who am I? You can find me on various social media platforms, and I'll be tweeting out various things as we proceed. A huge shoutout to all who helped me in devising quirky and fun aspects. Every now and then, you might have noticed I included some unusual touches during the presentation, and I did get permission for those. There were some peculiar reactions when I asked for consent, but I got it nonetheless. The fox tribute was to Pi the Lucky Stiff.

00:37:34.790 As I said, he was my first foray into Ruby, and he may have been a lot of yours too. This simply demonstrates that whimsy, silliness, and fun are still very much appreciated within the Ruby community; otherwise, I wouldn’t be standing here to speak. They would probably throw me out otherwise!

00:38:23.670 You may have noticed that Matz was hiding in one of my slides as himself. The Bundler tape gun was present in its rightful form, and Genius even made an appearance as a conductor for Ruby on Rails. We sought to add an amount of fun to our talk, and you might have seen the colorful lemur stickers scattered around the halls. But yes, there is a point—I'm not completely insane—at least, not yet!

00:38:54.990 My doctor advised me against any thoughts like that. Anyway, we put together a scavenger hunt, and you might come across little lemur tags floating around! If you search for that, you'll uncover the identities of all who have lemur stickers here. Now, if those with lemur stickers would kindly stand up to reveal themselves?

00:39:48.610 Take a good look around and see who these sticker holders are. Now, lemur sticker bearers, if you would please make your way towards the back of the room! I hope you all are swift, as they, too, are very fast! I must say, I can be rather dastardly!

00:40:20.860 Yet, that isn't all, as you may wonder what Red's up to next and where he's heading from here. You see, for those attentive, you might have uncovered a subtle little clue hidden in one of the slides. I must say, GTR is not an acronym I have encountered in any Ruby book before, but it will indeed make an appearance in The Year 2019, as Red Legume will return in a book published by No Starch Press titled, 'An Illustrated Guide to Ruby!'