00:00:11.639
Thank you so much for being here! We really appreciate it, and we're super excited to share our presentation with all of you.
00:00:18.600
Before we start, we just wanted to express how inspired we were to be here with all of you and to hear your stories.
00:00:25.439
We're also really inspired by all the amazing speakers.
00:00:31.380
Are there any speakers in the room? Let's give them a round of applause!
00:00:40.980
We've learned a ton from all of you and really enjoyed being here.
00:00:47.399
So thank you for coming to our talk! Welcome to "A Brewer's Guide to Filtering Out Complexity and Churn" or, as we like to call it, the coffee machine talk.
00:00:54.059
The goal of this talk is to show you how to remove the bitterness caused by complexity and churn in your application.
00:01:00.020
We have three key objectives over the next 30 minutes.
00:01:05.820
We want to show you how complexity sneaks into your application, how you can manage and notify it, and then how to remove it permanently.
00:01:12.479
First, let us introduce ourselves. Hello again, my name is Alan, and I use he/him pronouns. I've been using Ruby for about a dozen years and I'm originally from Seattle, so there's coffee in my veins! My favorite is a New Orleans cold brew with chicory.
00:01:34.259
Hi everyone, I'm Fito. I also use he/him pronouns and have 12 years of experience in Ruby. I'm from South America, but I'm coming from the San Francisco Bay Area this time.
00:01:51.840
I have to say, if there's one thing I love as much as Ruby, it's coffee! My favorite is a Ghirardelli dark chocolate mocha.
00:02:05.759
Alan and I work together at a place you might not expect, given that this is a conference for Ruby developers. We've actually heard this a few times during the conference—we work for Cisco Meraki, which is the largest Rails shop you've never heard of.
00:02:29.640
We've been friends for years and have worked together at three different companies over the last nine years. Along the way, we've seen a wide variety of codebases. Plus, we spend a lot of weekends writing code and drinking coffee.
00:02:44.819
Alan, you grew up around coffee, didn't you? Yeah, I did! Back when I was a kid, nothing fascinated me more than these mechanical coffee machines they had in my dad's office.
00:03:01.019
You drop in a coin, listen for the clink, make your selection, and then all of a sudden, the machine springs to life, hissing, clicking, and whirring. At the end of the ballet, there's this one last sound—the glorious, aromatic black liquid splashing into the cup. Ah, c'est magnifique!
00:03:15.720
These days, I'm a little bit more fascinated by the complexity we find in software. We both are. Like the coffee machine, there are all kinds of hidden complexity in software, but it doesn't always start out that way, does it?
00:03:28.680
Show of hands, how many of you have worked on a Greenfield application? Awesome! How did that feel? Great! Now, how many of you have worked on a legacy application? Even more of you! How did that feel? Yeah, that's been our experience too.
00:03:44.220
You know, Greenfield development feels great—it feels fast, and there's not any existing code getting in your way. But legacy code always feels harder to work with.
00:03:53.100
Why is that? We believe it has to do with software complexity. We think there's a threshold that a particular application will hit at some point, after which you only have two choices.
00:04:06.679
You can either live with the complexity as it grows and development slows down, under the impression that it will somehow magically disappear, or you can pause temporarily and reflect on the design of your software.
00:04:15.840
Then, you can reorganize it and speed up development again. We've seen organizations go both directions. When you take the path of living with the complexity, engineers often end up frustrated.
00:04:29.640
In our experience, they sometimes even begin to blame Ruby for the problem and look for alternative solutions. I don't know if there are any Rust fans here, but we have Rust enthusiasts in our office who constantly try to push our Ruby code in that direction.
00:04:50.700
It’s not Ruby's fault though—it’s the complexity that we ourselves added to the codebase. What we're going to do today is show you how to take that second path and remove the complexity from your system, so you can fall back in love with Ruby.
00:05:07.440
We'll build a little coffee machine application, and this application will have several features. We'll add them one by one, go back, look at how complexity sneaked into the application, and review each commit. Finally, we'll reorganize the code to demonstrate how to eliminate that complexity.
00:05:28.619
Are we ready? All right, let's get started! Who wants to build a coffee machine?
00:05:33.360
Fito: Let's do it!
00:05:39.540
Alan: Great! So, how does complexity sneak into software? The answer, of course, is one commit at a time.
00:05:51.120
Now, I'm going to go through these slides pretty fast just to show you the shape of the code as it grows, so don't worry about trying to understand exactly what's on the screen.
00:05:56.699
The code is made up anyway, and we're skipping tests for the sake of time. In reality, we will not be doing this without tests, right?
00:06:02.220
Let's get started! Here's the first commit in our coffee machine application.
00:06:09.419
At this point, the coffee machine does one and only one thing: it serves coffee. First, it dispenses a cup, heats the water, prepares the grounds, dispenses the water, and finally disposes of the grounds.
00:06:17.520
This works great, but it turns out not everyone likes coffee. To increase our sales, we're going to add tea.
00:06:29.640
Here, we've added a conditional to determine whether to serve coffee or tea, and in the process, we added some duplication.
00:06:38.220
The dispense cup, heat water, and dispense water steps are all duplicated between the two beverages.
00:06:44.699
Now that we have both coffee and tea in production, we're getting some feedback. Turns out the most requested feature is to add sweetener.
00:06:54.600
So let's go ahead and do that! Okay, so we’ve added the sweetener just after dispensing the hot water. Of course, not everyone likes sugar, so we're going to make it an optional ingredient.
00:07:05.880
We push this out, customers love it, and now they want cream. Let's give it to them! Since we already have a pattern for optional ingredients, let's dispense cream right after dispensing the sugar.
00:07:18.540
Some folks don’t like coffee or tea, so let’s offer them something else—like cocoa! Here we follow the existing pattern and add cocoa to the main if-statement, but there's no need to add sugar or milk since cocoa is already sweet and creamy.
00:07:36.240
Finally, who doesn’t like whipped cream on cocoa? Heck, I even like it on my coffee! So, let's add it. Whipped cream is an optional ingredient that no one wants on their tea, so let's add it after the other optional ingredients and exclude it when the customer is requesting tea.
00:07:55.020
Here we are, seven commits into this codebase, and we've already got nine conditionals in one method. At this point, it’s still relatively simple to understand and work with if you're the only one working on it.
00:08:11.520
However, procedural code like this does not scale well for a larger team, and one of the problems is that future developers will just keep adding more conditionals with each new feature, causing the complexity to skyrocket.
00:08:29.640
Our little coffee machine has been so successful that we've just been purchased by a big national soup chain, and they want us to add soup to our coffee machines. That's going to add a lot of complexity!
00:08:47.880
Let's pause here and evaluate where we are before we try adding more features. Alan, can you walk us through it?
00:09:02.760
Sure thing! So far, we've reached an inflection point in the life of our little application. But how can we tell what indicates that it's time to pause and rethink things?
00:09:19.680
The first hint is that we had to keep reducing the font size to get everything to fit on one slide. The second hint is that method length is actually a good indicator of complexity.
00:09:33.600
Even without measuring it, Sandy Metz has a rule about it. Sandy is the author of 'Practical Object-Oriented Development in Ruby.' If you haven't seen her talks, you should definitely check those out.
00:09:52.260
Sandy has a rule about method length: it can't be longer than five lines of code. This might seem really restrictive, but trust us; it’s a very important rule. In addition to method length, we also look at method complexity.
00:10:04.900
This is a quantitative measurement of how difficult it is to understand a piece of code. Our preferred metric is called assignments, branches, and conditionals.
00:10:12.600
There are other metrics like cyclomatic complexity, but this one is our preferred metric. The higher the number, the harder it is to understand.
00:10:21.000
We use a gem called Ruby Flog to measure this for us. It also includes some Ruby-specific metrics, so it’s a great tool for those who use Ruby. But how do we know what to do with it?
00:10:34.140
What’s a good score? What's a bad score for a method? Well, way back in 2008, a gentleman named Jake Scruggs, who wrote the Metric Fu gem, produced a list on his blog.
00:10:46.800
To this day, it’s the only list we've found that outlines good scores. We've been using these scores for years, and they truly do help drive you toward simpler software.
00:10:59.400
How do we think our little coffee machine fared? Let’s go back through it one commit at a time and look at what the complexity scores were.
00:11:09.960
Our first commit weighs in with a complexity score of 5.5. According to Jake, that’s awesome! The churn number there, by the way, is the number of commits that have been made to this file, which becomes important later.
00:11:25.920
Adding the conditional and duplicated code shoots the complexity up to 14.6. We’re not in that awesome zone anymore, but we're still in the good enough zone, so we’ll keep going.
00:11:40.020
After we removed that duplication, the complexity dropped significantly down to 10.9. This may seem like a good thing, but it's actually where things started to go wrong.
00:11:59.700
Notice how we've intermingled the two algorithms: you can no longer really see what it takes to steep tea or brew coffee; you have to read the whole thing and parse it.
00:12:12.480
This also sets a precedent for how future developers will extend this code by adding more conditional logic. From Flog's perspective, the math says that the complexity went down, but there is a valuable lesson here.
00:12:26.520
There is no magic metric that will tell you exactly what's going on with your code. There are tools that can inform you and help with your decision-making, but they don't replace the need to use your intuition about these things.
00:12:41.760
Pay attention to how hard it feels to add software as you’re going, and if it starts to feel like it’s slowing down, that’s a good time to pause and reflect.
00:12:59.400
Next, we added sweetener, which brought the complexity up to 13.5. Then we added cream, and it went up to 16.0. Adding cocoa pushed us over the good enough line to 21.3.
00:13:12.540
Finally, adding whipped cream increased the complexity to 25.5. If you look at the trend line here, you can see it's starting to curve upward, which is a good sign that it's time to pause and reflect.
00:13:28.320
Plus, we crossed Jake's threshold of 20 for good enough, so there you go! These are three ways to recognize when it's time to take a pause and reflect:
00:13:43.320
First, look at the method length. If you can’t see it all on one screen, it’s probably too complex. Sandy’s rule again is five lines of code.
00:13:56.040
Second, consider method complexity. Use a tool like Flog to produce a score; anything under 20 means you’re good to go.
00:14:09.480
Finally, use your intuition. How do you feel about the software you’re adding?
00:14:16.920
That’s how we knew we reached this inflection point, and it was time to start thinking about reorganizing this method—something Fito is about to do right now.
00:14:29.160
Fito: This sounds good! So we broke Sandy’s rule, crossed over Jake’s good enough line, and intermixed three algorithms—all of which sounds pretty dire. But is it possible to turn it around? Yes, absolutely! Let’s do it!
00:14:43.080
Here’s the method as we left it a moment ago. Now, while ‘drying’ code early can lead us in the wrong direction, let’s start by undrying this code to see if there are any missing abstractions hiding in plain sight.
00:15:00.500
We call this practice 'rehydration,' and it looks like this. Now, I know what you're thinking—this obviously increases duplication. But that’s what we need to do in order to find these missing abstractions.
00:15:19.080
Now we can clearly see each recipe, and since there's no overlap in the algorithms anymore, we can safely extract each one into separate polymorphic classes.
00:15:33.600
As you can see, we moved each recipe into its own class: one for coffee, one for tea, and one for cocoa. This structure has several benefits.
00:15:47.760
Each algorithm is now separate from the others, meaning that if you ever need to modify one of them—let's say to fix a bug or extend it—there's a much lower risk of introducing a regression in another one.
00:16:01.500
Plus, the main method is now much simpler. However, you might have noticed that there’s duplication between the classes. Methods like dispense cup, heat water, and dispense water are present in every class.
00:16:11.400
Now, we actually want that duplication because it makes understanding the complete algorithms for each beverage much easier since you only need to open one of these files to look at them.
00:16:25.380
We do not want to eliminate this duplication; rather, we want to ensure that there is only one implementation of each of these methods.
00:16:34.920
That’s what's meant by 'don’t repeat yourself.' It’s perfectly okay to call a method multiple times, but it’s preferable to implement that method only once.
00:16:48.600
Ruby provides multiple options for doing this. We could include a module, use composition, or introduce inheritance to define the methods in a base class.
00:16:55.140
In this case, because we’re using polymorphic classes and are unlikely to need these methods elsewhere in the application, we'll probably go with inheritance.
00:17:05.640
Now we’re almost done! There are just two remaining problems here. First, the main method is not open-closed, meaning that you'll have to modify the code to extend it.
00:17:17.460
Second, it has multiple responsibilities. Let's take a look at those responsibilities.
00:17:30.120
Its one true responsibility should be to prepare beverages, but right now it's also selecting which class to instantiate. That's the job of a factory, so let’s introduce one.
00:17:42.180
Okay, here we pulled class instantiation out into a factory class. Its only job is to choose which class to build based on the selected drink.
00:17:53.640
Now, the main method only has the remaining responsibility to prepare beverages, and it is now open-closed. This means we’ll never have to modify the main method's code again when we want to extend the functionality.
00:18:05.379
But you might have noticed that the open-closed problem just moved to the factory. We introduced another issue: the build method in the factory might return nil, which will cause the coffee machine’s main method to throw an undefined method error.
00:18:19.020
We can solve that by introducing a null object pattern. As you can see, the factory now returns a null beverage by default. So if we were to get a beverage type that we don’t support, it will just return the null beverage.
00:18:35.100
This class is just a simple beverage class with a prepare method that does nothing. Regarding the second problem with the factory, it’s still not open-closed.
00:18:47.520
This means every time we want to add a new beverage, we’ll have to modify the factory class and probably the tests that go with it.
00:19:01.980
We can solve that by using a different kind of factory. Here we’ve replaced the conditional-based factory with one that's based on a convention and a bit of metaprogramming.
00:19:11.700
The advantage of this approach is that it’s mostly open-closed. If we want to add a class name that doesn’t follow our convention, we’ll need to modify the convention, which would violate the open-closed principle.
00:19:23.700
For this reason, and because metaprogramming tends to obscure what’s happening, we actually consider this kind of factory an anti-pattern.
00:19:37.320
There are alternative approaches that are fully open-closed, and we would love to discuss those offline. We really enjoy that topic, so come chat with us later.
00:19:44.220
For the sake of this talk, let’s stick with our convention-based factory that’s mostly open-closed.
00:20:00.340
Now let’s take a look at where we are with complexity. Here’s that graph Alan was showing us.
00:20:17.520
Here’s where we left off after adding whipped cream. The next thing we did was hydrate the code, and look at that: it’s much more complex than the dry version!
00:20:35.760
But remember, the dry solution was hiding the fact that there were missing abstractions. So what we did next was pull the algorithms out into their own polymorphic classes, and this dropped the complexity significantly.
00:20:52.920
Finally, we extracted the factory object, and now the complexity is lower than it’s ever been! We are now well back in the awesome range.
00:21:04.680
In the main method we'll never need to change it again! Now let’s take a look at all the other classes we introduced in the process of refactoring.
00:21:14.460
There are six in total. The coffee machine has now settled down to a very low complexity of 3.9. The factory is next at 6.2.
00:21:27.900
Then there are the three beverages that are a little bit more complex but well within the good enough zone, so there’s nothing more to do with them. The same goes for the null beverage—it has zero complexity.
00:21:41.280
Now, there was a reason we did all this, right? We needed to add soup to our coffee machine without making it more complex.
00:21:54.960
So here’s where we left off. Let’s add some soup! There you go!
00:22:07.620
Adding soup did not require us to change any of the existing files; it just works. As long as the soup class is loading into memory, it will be available for the factory to instantiate.
00:22:21.240
In the future, adding a lower beverage will be as simple as just adding another class and the corresponding tests.
00:22:31.160
That's it! That’s a look at a very small greenfield application.
00:22:44.700
I know what you're thinking: your applications are obviously a lot bigger and a lot more complex. So how will you know where to start when you get back to work?
00:22:57.960
I’m glad you asked! So far, we’ve really only talked about method complexity and how monitoring it as the method changes over time will help prevent complexity from sneaking into your application and becoming painful.
00:23:12.720
But you can also use complexity and churn together—remember that number we mentioned earlier? Churn is the measurement of how many times that code has been changed.
00:23:28.200
We like to think of complexity as representing the amount of pain you'll experience the next time you work on that code, while churn represents how often you're inflicting that pain on yourself.
00:23:42.840
If there's something with high complexity and high churn, you're really just beating yourself up. Let's look at how to use churn and complexity to evaluate your entire codebase.
00:23:58.440
If you saw Ernesto's talk yesterday, this is going to look familiar—but we think it bears repeating; it’s very important. Like Dr. Tannenbaum said, you need to hear things two to four times.
00:24:14.040
So here we go! Find the areas in your code that need the most attention. You can plot file complexity and churn on a graph like this using a tool like Ruby Critic or Code Climate.
00:24:29.400
This kind of churn versus complexity chart was first proposed by Michael Feathers, the author of 'Working Effectively with Legacy Code.' Yet another book you should go read.
00:24:44.760
We use this kind of chart to locate areas of our code base that need intervention. To understand the graph, let’s divide it into quadrants and look at each one.
00:25:00.780
The lower left quadrant is the pain-free zone. These files are easy to change and easy to understand. In a healthy application, the majority of your files should live here.
00:25:16.740
The upper right quadrant is the painful zone. These files are hard to understand, hard to change, and prone to regressions. Being aware that these files are in this quadrant will help you make decisions about where to add new code.
00:25:31.740
You probably don’t want to add more code to these classes. In fact, if you have to touch them, it’s better to extract code rather than adding more to them. This will create simpler classes with a churn score of one.
00:25:44.760
In the lower right quadrant are the low complexity files that change frequently. It's possible these files are actually configuration data masquerading as Ruby code.
00:26:01.440
For example, there might be some JSON hiding in a .rb file. These files are better served as data files that your Ruby code can read in when needed.
00:26:15.780
Finally, in the upper left are the high complexity files that rarely change. These are what Sandy Metz refers to as 'omega messes.' An omega mess has a big, scary algorithm in it that really never needs to change.
00:26:31.560
Sandy’s advice? Don’t change it. Leave it alone! It’s not causing you any continued pain because there’s no churn, and it's so complex that changing it would likely break it.
00:26:46.680
These quadrants are helpful for thinking about this, but in reality, things look more like this. The red line represents the pain threshold. Anything in the pink zone is resistant to change and prone to regression.
00:27:03.300
This file here needs the most attention; it’s super complex and being modified all the time. Extracting hidden abstractions from this class will improve the complexity of your entire application.
00:27:19.020
But there's no need to tackle it all at once. Try to improve the code gradually as you touch the file. You may not want to start with that particular file—it’s both super complex and changing all the time.
00:27:33.480
Starting with a simpler file will give you a chance to practice these techniques we’ve taught you without the pressure of working on your most complex code.
00:27:48.300
So that's it! That’s the story of our little coffee machine application: how complexity snuck in, how we recognized it, and how we removed it.
00:28:02.040
Let’s wrap up with a few takeaways and some homework. Here are three things we'd like you to take away.
00:28:14.280
First, complexity will sneak into your application, and it happens one commit at a time. So, be vigilant! Pay particular attention to conditionals that you’re introducing into your code; they are often signs of abstractions trying to escape.
00:28:29.760
One of our friends, Josh Clayton, says: 'Don't make your code so dry that it chafes!'
00:28:44.160
Second, you can recognize complexity before it becomes painful. Look at method length, consider complexity, and trust your intuition. If methods are longer than five lines, if your Flog score crosses 20, or if you feel like things are getting slower, it’s probably time to pause and reflect.
00:29:01.920
Third, it's possible to back away from this complexity. Leverage polymorphism and factories to enable you to add new features without changing existing code.
00:29:18.480
Rehydrate your code by reintroducing some duplication to find those missing abstractions that can be pulled out as polymorphic classes.
00:29:35.880
Those are the three things we want you to take away. Now, here’s some homework.
00:29:48.540
First, find out what your average method complexity is using the Flog gem. I'm not going to tell you how to do that; you’ll have to figure it out for yourself—it’s easy!
00:30:11.100
Second, find out which file has the most churn in your application by using the churn gem—also easy. Finally, identify which class needs the most attention—this is the file with the highest churn and the highest complexity.
00:30:25.920
We bet you already know which class this is in your application, but confirm it. I see a lot of nodding heads; that’s awesome!
00:30:39.240
Please let us know what you find out! Here’s how to reach us. Feel free to tweet or reach out via the Slack channel we created for this talk.
00:30:56.520
We’ll answer questions there for as long as the Slack channel remains open. Also, if you want to walk through the coffee machine application, you can find it on GitHub.
00:31:08.420
Plus, you can check out other personal projects we have there, including a VS Code extension that shows you your Flog score.
00:31:16.680
The extension displays the average method complexity for the currently selected text, the method where the cursor is located, or for the entire file.
00:31:29.700
Just one last thing before we go: as we mentioned, we’re from Cisco Meraki. We are the largest Rails shop you’ve never heard of; we don’t build coffee machines, but we do build internet machines for coffee lovers.
00:31:42.900
Both Starbucks and Peet's Coffee are customers of ours. They use our software and hardware to connect their stores and customers to the internet.
00:32:02.760
We have a very old, 15-year-old Rails monolith with over a million and a quarter lines of code. It’s old, yes, super complex, yes, and it handles billions of requests today, supporting a multi-billion dollar business.
00:32:15.180
If you're interested in solving problems related to software complexity or large-scale Rails implementations, come talk to us! We’d love to chat with other passionate Rubyists who want to make a change.
00:32:30.240
That's it for our presentation! Thank you so much for coming! Here’s a list of our references and influences. Feel free to take a picture.
00:32:43.020
If you have any questions, come find us! Thank you!