00:00:00.060
Justin Searls is going to be talking to us about surgical refactoring. Please welcome Justin Searls!
00:00:05.490
I'm just reading his slides, so here I am to read some slides to you this morning. This talk is called 'Surgical Refactoring.' Can we bring the lights down so we can see the screen better?
00:00:19.650
Alright, cool. Good morning! Let’s start. My name is Justin Searls.
00:00:26.180
That’s me on Twitter, @searls. If you've seen my face before on Twitter, that’s what I look like. My email address is [email protected].
00:00:35.640
My last name in katakana is セアールズ (Searls). If you want to call me that, that’s fine. If you’re speaking in Japanese, you could say 'ジュスティン' (Justin) and all that.
00:00:54.570
I come from a company called Test Double, where we are software consultants. If your team is looking for additional developers who are really proficient in Ruby and JavaScript, as well as testing and refactoring, we would love to work with you. If you’re okay with people who mostly speak English, you can send us an email at [email protected].
00:01:24.689
A little bit about me: this is me eating some Yabaa tono miso katsu. I talk very fast when I'm nervous, and I am always nervous, so please forgive me.
00:01:38.810
I mean, we have plenty of time today, so if I'm speaking too fast, it’s okay to shout "moto!" You can tell me to slow down. Speaking of being nervous, I was very concerned about the screen size with the projector here this morning. I didn’t know if it was 16:9 or 4:3, but I think we landed okay.
00:02:18.640
This gave me an idea because Matz was talking about Ruby 3.x yesterday, and how difficult it would be to solve certain problems. I think we can solve it here today quite easily. It was a lot easier than he made it sound.
00:02:46.490
Speaking of Ruby, it’s a significantly successful programming language. In the early days, success looked like a lot of happy people building things for fun. People gave us tremendous positive feedback.
00:03:01.710
I remember when I started learning Ruby, everyone I knew who wrote Ruby seemed really cool. At the initial stages of any language, the marker for success is if it’s easy to create new things. One of the best aspects of Ruby is how effortless it is to build something new.
00:03:29.480
However, later success looks quite different. People become more critical as it gets popular. If the language is highly adopted and in use at work, the scrutiny increases. As time progresses, more money becomes tied to the existence of the programming language, which creates a different mood around it.
00:03:57.670
I believe the key to later success for a programming language is making it easy to maintain old things. Unfortunately, nobody enjoys maintaining legacy code. So my question today is: Can we make it easier to maintain old Ruby code?
00:04:25.470
Today, we are going to refactor some legacy code. The word 'refactor' stands out. You might define refactoring as changing the design of code without altering its observable behavior.
00:04:57.790
Essentially, you change the implementation but keep it behaving the same way. I perceive refactoring as changing in anticipation of a new feature or a bug fix to make that feature or bug fix easier to implement later.
00:05:13.720
Legacy code is the other critical phrase in the sentence. Legacy code has various definitions. Some people define it as 'old code' or 'code without tests.' Usually, we say legacy code in a pejorative sense—it’s code we don’t like.
00:05:47.660
But today, I’m choosing to define legacy code differently: Legacy code is code that we don't understand well enough to change confidently.
00:06:07.200
So today, let’s refactor some legacy code. Whenever anyone mentions refactoring and legacy code together, I feel that refactoring is indeed quite hard.
00:06:16.420
This is because you’re taking something that exists and making its design better, which requires a certain level of creativity that’s not always present. Refactoring legacy code is particularly challenging because it tends to be complicated.
00:06:42.060
As a result, it’s easy to accidentally break existing functionality for users, which makes refactoring feel dangerous. Legacy refactoring can cause teams to feel unsafe and increase anxiety. Additionally, selling the idea of legacy refactoring to managers and business people is incredibly challenging.
00:07:33.580
If we chart business priorities of our activities as developers against the cost and risk of our activities, new feature development falls in the top right corner—obviously high priority but also expensive. In the top left corner, we have bug fixes, which are high priority but relatively less expensive.
00:08:18.540
In the bottom left, low priority, but still relatively low cost, is testing. But where does refactoring fall?
00:08:28.500
Refactoring typically occupies the bottom right corner. It’s difficult to sell.re factoring to businesses; we don't need to convince them about new features if they're paying us a salary, and bug fixes are usually easy to sell because of their cost benefits.
00:09:14.400
Testing is also being successfully sold nowadays, yet selling refactoring ends up being challenging.
00:09:55.890
Refactoring remains difficult as it is hard to estimate how long it will take; you never really know how much work is necessary or its associated risks from the business’s perspective. Because when refactoring code, it is merely changing the implementation while the observable behavior stays the same, it’s basically invisible.
00:10:40.050
As we are often changing something that is very complex, we need to halt all other work in that code area to facilitate the process. As the complexity of legacy code increases, it generally means it had more importance: the business needs that code to process significant information. Thus, changing it becomes less certain and more costly.
00:11:40.000
As part of my 'Make Ruby Great Again' series of talks, I want to make refactoring great again. Of course, I thought about that line for a couple of seconds before realizing that refactoring has never truly been great.
00:12:26.000
In our quest to make refactoring easier, we can work on two areas. One avenue of improvement is selling refactoring to businesses at higher priority.
00:12:58.120
When we pitch refactoring, the image in their minds is often akin to road construction: we stop everything; no traffic goes through but money continues to seep out at the same rate.
00:13:35.380
This isn’t an attractive mental image for managers. We use several tricks to sell refactoring. One trick is to scare them into it, saying, "Hey, if you don’t let me refactor this now, we may have to rewrite everything later." But who can prove that?
00:14:15.980
We might also argue that future maintenance costs will be much higher, but that’s hard to quantify. It often feels unreal. Another strategy is to absorb costs through discipline and professionalism, integrating a little extra time for refactoring into each feature.
00:15:08.410
This strategy sounds fantastic, but it requires immense discipline that most teams usually lack, especially when there’s time pressure. Refactoring will invariably be the first practice to go, and most teams do experience constant pressure.
00:15:55.810
Another approach teams use is 'taking hostages.' In this strategy, the business sets the backlog priority, stating priorities for new features, but the team indicates, 'No, we need to do some refactoring before getting to feature three.' However, this approach can lead to an adversarial relationship.
00:16:43.000
Our message is essentially blaming the business for rushing us, which erodes trust because they are paying us a lot of money to write code and may think we are incompetent if we have to keep stopping to fix it.
00:17:44.360
Refactoring is challenging to sell, but we all believe in it. We’re probably not going to shift the culture overnight, so that is likely not where the solution lies. If we examine the other axis, which raises the question: Why is refactoring so expensive?
00:18:30.000
Whenever I refactor code, there is a lot of pressure on me. I have to complete much work quickly because other developers need my changes, and there is not much time allocated to it. Generally, our tools don’t effectively assist us with refactoring.
00:19:26.300
Most open-source tooling usually revolves around adding new features, which is more exciting. Since most developers are tasked with maintaining old code, you would think we’d possess better refactoring tools, but it hasn’t worked out that way.
00:20:02.720
For many, refactoring is a scary task. I’m on a mission to identify all aspects of software development that are frightening and find means to make them less intimidating because I feel anxious and scared all the time. If you're like me and constantly scared, you might want to buy my book that I'm working on called 'The Frightened Programmer.'
00:20:45.600
It’s not a real book though, as I'm too afraid to write it. So, what can we do to make refactoring easier in terms of cost? We currently do several things already: we have refactoring patterns, like those in Martin Fowler's book.
00:21:21.200
These patterns include explicit operations like extracting methods, pulling up, pushing down, and splitting loops. They have names because if you execute the procedure diligently enough, it becomes safe to pursue those specific refactoring operations.
00:22:11.380
It’s even safer if you possess good tools. For instance, one of my favorite aspects of Java is its expressiveness, which allows you to incorporate automated refactoring tools into your IDE with a relatively safe expectation that you won't break anything.
00:23:10.610
However, these tools aren't very conducive to handling more complex operations either. You can’t effectively take a complicated design and simplify it into a good design if you only stick to following prescribed operations.
00:24:01.040
Another technique called characterization testing, popularized in Michael Feather's book 'Working Effectively with Legacy Code,' provides excellent advice regarding this.
00:24:58.200
This method treats the code as a black box. You set up a test harness around it, feeding it inputs and recording the outputs.
00:25:31.960
You send inputs into the black box and capture outputs without judgment. The aim is to create a harness that will reassure you that as long as the tests pass, your changes have been safe.
00:26:18.460
Once you establish that test harness, you can then aggressively refactor the code into new units or new objects, eventually backfilling unit tests that understand what the code does and explicitly indicate their intentions.
00:27:10.740
However, you must write all these characterization tests first, which takes considerable time. You also need to write new unit tests for those. In summary, that’s a lot of testing.
00:27:54.640
When you finish your refactor, you typically delete your characterization tests. But if you have a lot of legacy code, you might want to maintain all the test coverage you can. After spending considerable time writing a test, you may not want to delete it.
00:28:43.680
Because the process can be draining, it's tempting to quit halfway through—it’s exhausting and not ideal. Recently, a technique resembling A/B testing has become popular for legacy rescue.
00:29:20.420
Essentially, you can write a new implementation beside the old one and put a router in front of it. This allows you to send a portion of the traffic to the new code while the majority still hits the old code.
00:30:06.860
Jesse Toth, who may be here today, has an excellent gem on GitHub called Scientist, which helps facilitate this process. However, it does require a great deal of sophisticated monitoring, logging, and data collection to ensure that any changes are indeed safe.
00:30:51.000
Moreover, it's only suitable for business domains where it is safe for transactions to fail; it could work for GitHub but might not be appropriate for banks handling financial transactions.
00:31:31.420
If you think of it as a spectrum, characterization testing lies on one end, and Scientist on the other. Michael Feather's approach is good for development but fairly painful for testing and offers no solution for staging or production.
00:32:17.440
Scientist lacks clear solutions for development and local testing but is beneficial in staging environments and potentially overwhelming when it comes to production due to the amount of data it generates.
00:33:04.240
What if we had a tool genuinely competent in all four stages of a refactoring's lifecycle, from planning to completion? That was the question I posed to myself when submitting to speak at this conference. After months of thought and remaining frightened, I decided instead of writing numerous slides explaining how to refactor well, I would write a new gem.
00:33:44.620
Although I speak in English—and sometimes people find my speech challenging to follow—Ruby is the language we all share. So let's discuss Ruby for the rest of the talk. I used TDD—a tactic I refer to as Talk Driven Development—for this project.
00:34:32.400
The tool I wrote is a new gem called Suture. Suture is the stitching used to close a wound after surgery, making it an apt metaphor for surgical refactoring. You can find it up on our GitHub at Test Double.
00:35:45.000
The page looks like this, and you can install the gem just like any other gem. The metaphor here is that refactoring is like performing surgery. Surgeries serve a common purpose: helping us recover. We want to take something intimidating and make it feel safe.
00:36:36.730
They require thoughtful upfront planning and flexible tools because each refactor may involve similar information. This same information can facilitate development, testing, staging, and production, leading to a consistent process definition.
00:37:31.000
First, we need to plan the refactor, then identify the seams we are going to cut, record the interactions of the old code path, and automatically validate in a testing environment that we can reproduce all those recordings.
00:38:04.140
Finally, we get to refactor or re-implement the code, verifying that the new refactor behaves as expected when matched against the original recordings in a staging environment.
00:38:54.350
In production, we can use this information for a fallback mechanism to recover any errors in the new code that we couldn't anticipate otherwise. The last step is to delete Suture. Just like stitches, Suture shouldn't remain in your gem file indefinitely—only during legacy rescue.
00:39:47.490
For planning today, we will perform two bug fixes. The first is a Calculator Service that erroneously handles negative numbers. It is structured as a pure function, meaning it should behave predictably. In this controller method, we create a calculator and call 'add' with the two operands, setting the answer to 'result.'
00:40:35.920
The implementation structure indicates that for whatever number of times the right operand is passed, it loops accordingly, adding one to the left and returning it. This is where the bug lies because it only works for positive numbers. Legacy code is indeed messy, but I'm sure yours is too.
00:41:16.890
Next, we zoom back out to create our seam, which is the point where we intend to branch between the new and old code—this is the call site, and the most logical location to implement the division.
00:41:59.839
Our next bug fix involves a Tally service to invoke the calculator multiple times before asking what the sum of all the numbers was. This function, in contrast, requires an object mutation over time.
00:42:45.660
In this method, we once again instantiate a calculator and call a tally function for each value in an array. If we examine this code closer, we see that it includes a ridiculous loop over the numbers, counting down to zero and implementing addition only if the number is even.
00:43:36.930
Thus, the tally function only multiplies even numbers, while any odd number feels ignored. So, our mission today is to correct this error. This call site requires more extensive adjustments because it also depends on a value of a 'total' instance variable.
00:44:28.250
Next, we need to modify our seams. In the pure function case, we will create a Suture instead of directly calling 'calc.add.' We will instruct Suture to create a seam called 'add', passing it the necessary arguments such as an array of operands and a reference to the callable add method.
00:45:25.100
In a mutation case, we will have to define a different Suture creating the tally function. The arguments used for 'tally' will include both the calculator and the numeral to be summed. The overall idea is that functions vary based on their context and carry different implications.
00:46:15.540
Pure functions are simple; we treat them as black boxes, passing a handful of arguments to receive a return value. When the same values are reintroduced, we can expect the same output.
00:47:03.700
In contrast, mutating functions compound the complexity because their outputs can shift unpredictably based on state. The tally function's result varies according to the total state representing the 'calculator'.
00:47:57.800
Finally, once we establish that our recording operates effectively, the next task is ensuring we recreate the outputs. For pure functions, we will write a simple verification test by retrieving the recorded instances and comparing their outcomes.
00:48:35.800
In the mutation context, we would replicate a similar structure but must ensure that the lambda behaves identically for accurate results. If they differ, we’ll learn from it, leveraging the error messages generated by Suture to guide us.
00:49:17.160
Just as a reminder, we’re not just cleaning up the repository; it’s about making growth at a manageable pace while ensuring that all our tests remain relevant. We believe in refactoring but know we need to backtrack, decompose, and understand the function before outright replacing it.
00:50:15.740
When utilizing Suture, we should still follow a cautious approach, particularly during the implementation in production environments. We set up a fall-back mechanism that will allow us to revert to the old code if necessary, enhancing safety measures.
00:51:11.200
Having tested the function in production with a similar result should instill confidence across both the testing and current execution environment. As we wrap this session, I implore you to delete Suture, creating an analogous experience to removing stitches once the wound has healed.
00:52:01.170
We can assess a routine code structure and clean things up. So far, we’ve successfully navigated through this comprehensive, thorough approach to refactoring software with the help of Ruby’s dynamic nature.
00:52:47.360
Suture is released, and although I developed it, I haven’t shared it beforehand during projects or In the laboratory; I’m looking forward to utilizing it. I have released the 1.0 version today, post confirming its efficacy.
00:53:45.310
I would love for you to experiment with it. The GitHub repository is Test Double/Suture, and I'm going to ensure that the API remains stable indefinitely moving forward. Working together with tools like Scientist and Suture will help reduce the stress surrounding refactoring.
00:54:34.680
In the long term, I believe we can make Ruby and software more sustainable for businesses through easier maintenance and management of legacy code.
00:55:19.490
One more personal note: I'd like to share how I met Ruby years ago while studying. During my homestay in Chiba, I discovered a book named 'Programming Ruby.' I had only seen Ruby in America and was thrilled to browse through its pages. Talking to friends back home, I found excitement and appreciation for Ruby, especially as Rails was gaining traction.
00:56:20.760
Over the years, I have continued with Ruby and Rails, widening my knowledge, and I thank you for this opportunity to share my experience with Ruby. Thank you very much!
00:56:57.000
I would appreciate it if you would follow me on Twitter, and perhaps we can connect. My wife and I are heading to Kansai, and I'd love to meet up for coffee or dinner—especially in Kyoto or Osaka.
00:57:14.440
Sure, I can answer that question! :) Thank you so much for your attention!
00:57:26.990
In a subsequent question, you asked if Suture identified any faults in your code and whether you’ve tested it in production. I mentioned using it in 20 projects with high expectations, and though I miss my programming days, I feel confident in Suture’s reliability.
00:59:00.400
If something goes awry, I assure you I will stay up all night fixing it. I would therefore not provide a warranty.
00:59:14.440
Another participant inquired whether Suture’s usage would comprehensively cover CI migrations challenge or if it is simply employed locally. I confirmed that while it works locally, it performs well in CI environments too!
00:59:57.000
Thank you! I hope you enjoy using it!
01:00:02.640
Could Suture develop tooling that identifies global variables for easier management? The response indicated my consideration to sequence various calls to tackle their ramifications better.
01:00:20.890
Potential future implementations may evolve to craft new tools enabling monitoring of network request behaviors.
01:00:31.780
An attendee mentioned the scenario where a legacy application may excessively utilize global variables or network conditions hindering refactoring efficiency.
01:00:57.940
Thank you for this wonderful exchange, and I appreciate your investment in this talk!