Ancient City Ruby 2013

Test Driven Development: A Love Story

Practicing Test Driven Development (TDD) is like falling in love. It may first seem like all your development problems will disappear. However, it's not all unicorns and rainbows. You have to work at it, and keep working at it, for the rest of your development life. It is hard, and it's natural to question whether the value is worth the effort.

So why do it? Why would you bother going through all that trouble, dramatically changing the way you code, learn new domain specific languages, and initially slow down the rate at which you produce code when you have no time to lose?

This talk will answer the "why" by sharing my experience of passing through the five stages of grief (denial, anger, bargaining, depression, and acceptance) as I learned TDD, and how acceptance grew to love.

You will walk away from the talk with techniques for maintaining and strengthening your relationship with TDD. Test frameworks and languages may come and go, but the fundamentals and value of TDD remain.

Ancient City Ruby 2013

00:00:00.000 a couple of years ago I started a dream job I would be doing what I love coding
00:00:07.140 and building software and what's even better I would be doing it in Ruby the
00:00:12.929 first six weeks of this job were absolute lists I was working on a relatively new codebase a co base that
00:00:19.680 had some rough patches to be sure but the vast majority of it was very maintainable and very understandable now
00:00:27.570 during this time of bliss I did hear whispers of another code base a code base so terrifying even the
00:00:37.530 bravest developers fear to tread there this was 40,000 lines of gnarled nasty
00:00:45.629 snarled largely untested legacy code now I figured I'd be introduced this code
00:00:52.079 base in due time piece by piece but as always the unexpected occurred
00:00:58.190 the only other full-time developer at the company left and suddenly this
00:01:03.660 legacy code base was my responsibility and mine to tend my days of bliss turned
00:01:10.229 into days of receiving a frantic bog report panicking writing fix as fast as
00:01:15.570 I could deploying that fix and breaking more things and panicking again this cycle
00:01:21.570 will repeat over and over and over test-driven development wasn't a part of
00:01:27.180 the equation at this point I felt it was taking all my time and energy just to
00:01:33.030 keep this system above water I didn't see how I could incorporate incorporate tests into that now scare as I was of
00:01:41.670 the bugs that I did see what scared me the most for the ones I knew I didn't
00:01:47.070 see the silent bugs lurking in the depths of this legacy code now during
00:01:54.270 this time I once woke up screaming I had dreamed I've been working on this legacy
00:02:00.299 code based on my laptop when all of a sudden the code turned into a mutant bat and flew directly into my face scare the
00:02:08.250 heck out of my fiance this code was invading not only my
00:02:13.940 conscious mind it was seeping into my unconscious mind as well it felt dark it
00:02:22.010 felt hopeless it felt like an endless night that I would never escape but dawn
00:02:29.000 did break I don't wake up screaming anymore I don't dread opening up my inbox in the
00:02:36.110 morning and having to sift through the deluge of errors and exceptions that came in overnight I used to feel like I just coded to
00:02:44.840 apply band-aid fix after band-aid fix since lazy codebase I actually build
00:02:50.239 things again I do what I love again I want to tell you a story a story that
00:02:58.670 starts with love a love for code a love for building software now this love is
00:03:06.230 severely tested when I inherited this legacy code base and I actually passed through the five stages of grief denial
00:03:15.250 anger bargaining depression and finally acceptance
00:03:20.959 I also want to share how this grief turned into love again a love that is
00:03:26.750 more passionate and stronger than it ever has been before test-driven
00:03:32.780 development was the key to learning to manage my grief moving through my grief and learning to manage this legacy code
00:03:39.109 system it didn't happen overnight and it certainly wasn't easy even now after
00:03:44.930 working as a test-driven developer for some time now it's still not all unicorns and rainbows I still have to
00:03:52.250 work at it there are still rough patches and there is change but the difference
00:03:58.819 is now those changes are not insurmountable so who here has had to
00:04:05.419 deal with legacy code just about every hand went up does that code ever scare
00:04:11.209 you does it haunt your dreams
00:04:16.670 I want to share my story to help you move beyond the grief that anger
00:04:23.030 associated with legacy code and rediscover the joy in coding you can
00:04:28.370 tame a legacy system I'm here to share how after I inherited this legacy code I
00:04:37.040 found myself at the first stage of grief denial now this legacy code base had some tests
00:04:43.700 but the vast majority of it was completely uncovered I was relatively
00:04:49.100 new to test-driven development when I started this job and I'd only tried it on green field applications I thought
00:04:56.210 that it wasn't the time to try adding it to an existing application I was getting
00:05:02.450 errors and exceptions left and right I felt I had to stabilize the codebase
00:05:07.940 first and then worry about tests to do this I resolved a stick to manual
00:05:13.070 testing for the time being I would code afix test it manually deploy it if that
00:05:19.790 fix broke things I would code another fix deploy it again and if that broke more thing is I would code another fix
00:05:26.300 and deploy it yet again the problem with this approach was I had no way of
00:05:31.310 knowing if I changed one class whether that change would affect two other classes three other classes even ten
00:05:40.190 other classes with every deploy I was flying blind and I knew this but I still
00:05:48.500 thought I would add my tests when the crisis passed when I had a moment to
00:05:54.440 breathe I convinced myself I just needed to hold out a little longer
00:05:59.680 surely things we get easier soon surely then only then would I have time to add
00:06:06.020 tests the truth though is a magical better time in the future never comes
00:06:13.810 every untested line of code I committed only made the system worse and spiked my
00:06:19.160 stress level even higher Kent back writes and test driven to the
00:06:24.190 test-driven development by example that the more stress you feel the less testing you will do
00:06:32.720 the less testing you do the more errors you will make the ideal time to write
00:06:39.720 tests is not some magical time in the future the ideal time to write tests is
00:06:44.760 always now now I can only stay in denial for so long I knew things were falling
00:06:51.270 apart I even knew that adding automated tests would be a way to make it better but I didn't know where to start any bug
00:07:00.450 fix any new feature I added depended very heavily on untested legacy code
00:07:08.120 Ryann tests seemed impossible how can I write tests for someone else's code even
00:07:15.030 someone who had left the company years ago how would the code even gotten to
00:07:20.100 the state I honestly started to get angry now has anyone else looked at
00:07:27.510 legacy code and felt angry did you think
00:07:32.820 to yourself what was this developer thinking I would go in circles in my
00:07:40.200 mind ranting and raving about the sins of past developers why did I have to
00:07:45.750 clean up their mess why was it my responsibility fortunately someone
00:07:52.350 pointed out to me that the truth is the past developers were probably just as panicked as I was does anyone really
00:08:00.360 write bad code just for the sake of writing bad code there's anyone write
00:08:06.000 untested code just for the fun of it I could rant and rave about the past all I
00:08:12.870 wanted but it wouldn't accomplish anything the truth is you simply can't
00:08:19.260 change the past the I couldn't go back in time and write the legacy code better
00:08:24.930 or prevent it from being written the way it was I needed to move forward it
00:08:31.020 turned out the thing that needed to change the most wasn't actually the code the thing that needed to change most was
00:08:37.740 me I needed to take ownership of this code as ugly as it was
00:08:43.080 I could curse the past day in and day out but the only thing I could do was
00:08:48.090 affect the future the only thing I could do was make it better from here on out
00:08:54.830 now it is possible to add tests to legacy code but it does take time
00:09:00.620 the problem with code that was written without tests in mind is that often it's
00:09:06.030 untestable to be testable it needs to be refactored and refactoring a working
00:09:13.380 codebase particularly one that doesn't have test coverage to begin with is always risky no rapin writes in rails
00:09:22.200 tests prescriptions take small steps that can be verified I couldn't add test coverage this 40,000
00:09:31.530 lines of code all at once this wouldn't be practical from either a technological or a business standpoint but there were
00:09:39.480 certain steps that I could take any new code must have tests now when I read
00:09:48.270 rails tests prescriptions I found a pretty good piece of advice which was to separate my new testing code put it in
00:09:55.080 new tested classes and methods and call those tested classes and methods from within my legacy mess now this is
00:10:02.910 actually the first step in refactoring that legacy code separating things out
00:10:09.350 bugs in code must be reproduced with a test now the way I found to do this
00:10:16.710 because that AWS honestly is a little is much easier said than done was to use the bug report now when a user submits a
00:10:23.340 bug report if they're nice though include steps to reproduce that bug what
00:10:28.740 I did was use these steps that the user used from the graphic user interface to figure out how to reproduce that bug on
00:10:35.820 a code level so let's say my user gave me a bug report and it stayed with that
00:10:43.050 one that user visited a form let's say it's a form for a new record in the database a new widget or something when
00:10:49.560 they visited this form they would leave a certain field blank then they would
00:10:55.200 click Submit now when they did this the application would crash completely how to do this in
00:11:02.600 a test was create a new object chances are a forum is probably rendered by a
00:11:08.029 controller particularly in rails code one of the things that controller action does is create a new object in the
00:11:13.820 database I would then leave a field on that new object blank save that object and when I
00:11:22.160 did this in a test I found the test returned an exception
00:11:27.459 so this beg the question what should my code do should it crash completely when
00:11:33.830 a user just leaves a required field blank or should it return a useful error
00:11:39.020 to that user I tend to lean toward the useful error so to do this in a test I
00:11:45.770 did this using the r-spec test syntax I create a new object assign that to a
00:11:51.589 variable I set required field on that object to nil I saved that object and
00:12:00.279 then I specified what I wanted my code to do I wanted that new object to return
00:12:05.930 an error and I wanted to return that error to the user saying that where certain required field was blank I then
00:12:12.860 wrote the code to make this test pass you can use tests to learn about legacy
00:12:20.060 code my tests let me interact with my legacy code directly on a code to code
00:12:26.390 basis now separate documentation might lie comments in code might lie but
00:12:33.170 exercising the code itself doesn't lie with every new test I added I learned
00:12:39.470 more about this legacy code base when you're adding tests to existing code it's okay if those tests have an
00:12:46.459 extensive set up the point is to learn about the system the more I learn about
00:12:51.470 this legacy system the more I saw how I could change it for the better now once
00:12:58.970 I realized I could add tests to my legacy code and use these tests to improve the system my anger dissipated I wasn't done with
00:13:07.310 my grief yet however I moved on to bargaining now I embrace
00:13:13.670 the need for tests but I was still in the middle of a crisis bugs and
00:13:18.980 exceptions were still coming in left and right urgent bodies that needed to be fixed now I decided to fix the problem first
00:13:29.480 write my code first and then write my test after all how could I write tests
00:13:34.610 for code when I didn't know what the code was supposed to do on the first place now this is what I envisioned my
00:13:40.400 workflow look like I would write my code and then write my tests it's a two-step process in an actuality however this
00:13:49.220 process had many more steps I would write my code manually test that code
00:13:55.930 modify the code manually test those modifications again write an automated
00:14:02.750 test realize I had to modify my code to make the code testable manually test the
00:14:09.650 modifications to those code and then modify my tests to work this actually
00:14:16.010 wasted a lot of time when you test
00:14:21.140 laughs last it inevitably leaves holes in your test coverage now when I did get
00:14:26.810 to writing tests I wrote them fully expecting them to pass when they passed
00:14:32.450 I deployed but the problem with this approach was that sometimes after the deploy the code would break but the test
00:14:39.140 still passed this was a red flag that something was very wrong in my approach
00:14:45.709 to testing I found this workflow works a
00:14:50.810 lot better I'd write a failing test make the test pass and then refactoring
00:14:57.529 refactor the code chances are that looks very familiar to all of you this is also known as red Rian the failing test Green
00:15:05.450 making the test pass and then we're factored in growing object-oriented
00:15:11.630 software guided by tests Steve Freeman and Nate price lay out the golden rule of test-driven development never write
00:15:18.860 new functionality without first writing a failing test it's called
00:15:24.389 test-driven development for a reason when I test first I'm forced to clarify
00:15:30.850 my intention to find exactly what I want my code to do then after I have a
00:15:36.790 failing test in place I write code to do that to do my intention and to do only
00:15:42.069 that Robert C Martin in his clean coder
00:15:47.350 screencast series says that testing is about trust trust that when you write
00:15:54.790 code in all your test paths and you deploy that code it won't break anything in your system the only way to have full
00:16:02.999 100% trust is to have every line of your production code be tested now the only
00:16:09.759 way I found to come even remotely close to this number is to write my tests first when I read my tests first and
00:16:16.809 then make that test pass I know my code is covered by tests it wouldn't have
00:16:21.970 passed the test otherwise it was actually pretty disheartening when I
00:16:28.209 realized this when I realized that my attempts at bargaining and my attention doing things on my terms had failed I
00:16:36.689 moved on to the fourth stage of grief depression I felt hopeless again sure I
00:16:46.029 could write tests first from now on but so much of the legacy code including now code that I myself had written would
00:16:53.139 still be so bad what was worse that even tests I had written or that other
00:16:58.509 developers had written in the past ten had some of those tests were unreliable
00:17:04.720 there were gaping holes in their coverage because those tests had been written last I found myself asking
00:17:11.740 what's the point what's the point of adding new tests when so much of the
00:17:16.959 code an existing test would still be so terrible I honestly felt like giving up
00:17:22.809 I felt like writing off this legacy code base as a lost cause that I
00:17:28.419 unfortunately still had to use now it sometimes seems easier to do this
00:17:34.650 just to give up and resign yourself to living with the legacy mess but the
00:17:40.180 truth is it's not production code doesn't stay in stasis you will have to
00:17:49.510 add things and you will have to change things to a system that's still in use
00:17:54.570 when faced with a legacy mess you have two choices one you can add to the
00:18:01.240 legacy mess let the code get worse and let it rot and collapse in on itself or you can
00:18:08.290 move forward you can draw a line in the sand and make it better from now on
00:18:14.730 every line of testing code is a reliable piece of code every reliable piece of
00:18:21.490 code is a gift to yourself to your teammates to your users now by this
00:18:27.850 point in the story I was not the only developer at the company anymore there are several other people also working to
00:18:33.670 tame this legacy mess every piece of reliable code was a step out of the mess
00:18:39.990 every piece of tested reliable code is a beacon of hope in the dark snarl of
00:18:47.530 legacy code by contrast every line of
00:18:53.020 untested code is an unreliable piece of code every piece of unreliable code only
00:19:00.190 makes things worse for yourself for your teammates and your users the legacy code
00:19:06.910 may have been bad then but if I kept on committed unreliable untested code it
00:19:12.490 would only get worse like it or not I now owns this code I was steering its
00:19:21.100 direction as painful as it was as tempting as it was to give up it wasn't
00:19:28.900 hopeless it's never hopeless to start doing the right thing
00:19:38.620 now the right thing to do to save my sanity and the sanity of my teammates
00:19:44.000 and my users was to test drive my code from now on it was to move forward at
00:19:51.740 this point I found myself at the final stage of grief acceptance I finally came
00:19:58.790 to terms I knew I would survive this legacy code base and that test-driven
00:20:04.130 development of ival it was time to leave
00:20:09.320 the past exactly where it belongs in the past and move on to a brighter future
00:20:15.020 code is at this point that I realized
00:20:21.290 the true benefits of test-driven development one the most obvious is that
00:20:26.480 tests prevent breaking production code doesn't live in a vacuum as I said
00:20:32.540 before you will have to add things and you will have to change things change is inevitable what's not inevitable but we
00:20:41.270 can control is how painful these changes need to be now the most painful changes
00:20:48.110 for me are when I make a change and it breaks something completely unrelated in the code has anyone else had that
00:20:53.450 experience I see a lot of heads nodding when I have a full test suite especially
00:21:01.550 one written with tests first it prevents this from happening now even when I run that test suite and a test fails
00:21:08.870 unexpectedly if I'm not on that test suite often chances are I'm going to know exactly what change broke that test
00:21:16.210 the best time to fix something is directly after you break it now this
00:21:22.760 meant I could keep coding rather than constantly chasing down bugs testing it
00:21:28.430 turns out didn't just keep my code from breaking it kept me from breaking as well tests or documentation they're
00:21:38.600 living fully functional documentation that's entwined with your code now
00:21:44.150 rather than keeping my documentation in a separate place say a wiki or god forbid a word
00:21:50.660 my tests were embedded directly in my code I couldn't change one without changing the other the documentation
00:21:57.630 excuse me was embedded directly with my code now some of the hardest cases to debug are when the originals developed
00:22:04.650 original developers intent is unclear how can I fix a piece of code when I
00:22:10.530 have no idea what that code was supposed to do in the first place test document
00:22:17.190 the intent of the developer they state in no uncertain terms that when the code receives a certain input it should
00:22:24.360 return a certain output or take a certain action so let's look at some examples of this I have my first test
00:22:31.470 saying that when I call a method called tax rate and pass in the argument Seattle it should return point zero nine
00:22:38.430 five the tax rate in Seattle is nine point five percent I don't actually know
00:22:43.590 what it is in agustin this tells me exactly what this method is expected to
00:22:50.070 do exactly what input it should receive and what output it should return with
00:22:55.170 that input now the next one let's say I want to call a controller action I want
00:23:01.530 to say that controller controller action assigns a variable called new object as
00:23:06.810 a new record in my database it should not be an existing record it should be a new record I can also say I expect that
00:23:17.250 a certain method which saves this record to our database will change the table count by one not by two not by zero it
00:23:25.260 will add one record to that table now these test State exactly what my
00:23:31.500 intention is with my code test-driven
00:23:36.720 code is better code test-driven code is modular loosely coupled and has smaller
00:23:44.580 methods it's really hard to test something that doesn't have these attributes now these are the hallmarks
00:23:51.300 of good code testing Jason doesn't just tell me to write clean code it compels
00:23:57.960 me to write clean code code that is maintainable that is more readable and all around better crafted
00:24:07.130 finally tests remove fear when I have tests for my code I can refactor that
00:24:14.370 code fearlessly I can try new ways of implementing the same functionality maybe something I heard at a conference
00:24:21.240 or something I heard in the news or a user group or on a newsletter I can constantly make my code better and I can
00:24:28.440 do this without fear
00:24:34.460 even with this gnarled nasty legacy mess I could still improve the code I can
00:24:42.300 move forward rather than staying stuck in the past it will be a step by step piece by piece process but I knew this
00:24:50.040 code would get better I would make it better now I wish I could say that after
00:24:59.309 I pass through these stages of grief this legacy code and I rode off into the sunset and lived happily ever in
00:25:04.950 test-driven developed bliss but that wouldn't be reality I would be lying if
00:25:14.910 I said that testing ever truly gets easy it does get easier and you will see the
00:25:20.460 benefits very quickly but much like love itself it's something that you have to
00:25:26.100 keep working on now about a year ago I found myself completely baffled when it
00:25:33.450 came time to test drive code that interacted with an external API I knew I
00:25:38.820 couldn't connect to this external API every time I ran my tests this would be impractical at best and would get me
00:25:45.960 banned from the API service at worse I was convinced I had to figure out this
00:25:52.260 out myself if I couldn't figure it out myself it had to be impossible fortunately a
00:26:00.540 colleague pointed out to me that I didn't have to go at it alone when testing gets hard and it will it's
00:26:08.309 okay to consult a teammate it's okay to do a search on Stack Overflow or a search in Google once I and to learn
00:26:16.500 from experiences of others once I managed to move past my ego and actually
00:26:21.750 Google the problem it turned out that many others had already solved the problem of test-driving code that
00:26:27.660 interacts with an external API now what
00:26:32.880 I expected my code to do was call a method on the API service that API
00:26:38.610 service would return a certain response and my code would handle that response it turns out that for testing purposes
00:26:46.770 it was actually irrelevant whether my code connected to the live API the way
00:26:54.480 to do this was using mocks and stubs now mocks and stubs are hard it took me a
00:27:00.360 long time just to conceptually figure out what they actually do a stub is a
00:27:06.600 stand-in for an object called by your code it receives messages from your code
00:27:11.850 when it's under test and returns scripted responses responses that you tell it to return so under test
00:27:20.760 conditions my code would call an API method that call will be received by the
00:27:26.490 API stub the stub would then return a scripted response response that I told
00:27:31.679 it to return now marks are specialized
00:27:37.320 types of stubs in eloquent Ruby Russ Olsen who just spoke before me I didn't
00:27:42.570 actually know that until I started speaking points out that mocks are stubs with attitude so amok adds a lot acts a
00:27:51.809 lot like a regular stub my code will call an API method that method will be received by the mock or that call will
00:27:58.080 be received by the mock and the mock will return a certain scripted response now it's special about a mock as opposed
00:28:05.610 to a regular stub is that a mock knows exactly which of its methods should be called and with what arguments if my
00:28:13.080 code calls an unexpected method my mock will fail the test mocks are much more
00:28:21.570 specific than regular regular stubs if their expectations are not met exactly they will fail the test now the
00:28:31.650 answers were out there I just needed to reach out it wasn't just being my code
00:28:36.660 all alone in the universe there's an entire community an entire world developers all working on who maybe
00:28:44.220 already resolved that already solved the same problems that I was working on just
00:28:51.150 as I need to constantly learn and adapt the way I code I constantly need to learn and adapt the way I test
00:28:58.070 test-driven development is evolving new test frameworks come and go new best
00:29:03.810 practices are being uncovered in regards to testing every day it seems I recently
00:29:09.240 read practical object-oriented design and Ruby by Sandi Metz who also happens to be in attendance the chapter on
00:29:16.200 testing opened my eyes to how I can make my tests not only have better coverage
00:29:21.540 but be smarter it introduced concepts that I never even conceived of even
00:29:27.450 after working as a test-driven developer for some time I know that I will never
00:29:33.930 stop learning about testing just like I'll never stop learning about coding the way I code and tests will change
00:29:41.550 over time but even with these changes the core benefits of testing preventing
00:29:47.280 breaking acting as documentation and compelling you to write better code remain the same now my relationship with
00:29:55.560 test-driven development is not perfect but no relationship weather between two
00:30:00.960 people or between a person in their technology is ever going to truly be perfect that's what makes it worth doing
00:30:07.740 that's what makes it rewarding so test-driven development change the
00:30:13.920 way I code it gave me new tools for dealing with a legacy mess but even more
00:30:19.830 than that test-driven development changed me it transformed me from a
00:30:25.170 developer who panics in the face of legacy code to an actual craftsperson a crass person who cares not that my
00:30:32.700 code just works for the current moment what you designs code build things to last a class person
00:30:41.330 doesn't just rant and rave about legacy code but takes action to make it better
00:30:48.700 testerman development is developing beyond myself beyond my immediate needs
00:30:53.929 the present moment it's developing my code to handle change without pain it's
00:31:00.380 providing a safety net and a guided path for any developer who might come after me now change is inevitable we hold the
00:31:09.860 power to make that change something we can manage maybe even enjoy rather than something we dread test-driven
00:31:17.809 development empowers us to do this I better understand why I do what I do now
00:31:25.700 I love to build things that last that people can add to and change over time
00:31:33.130 now this love was severely tested when I inherited this legacy code I had to pass
00:31:39.590 through the five stages of grief but I did rediscover that love and that love is stronger than it ever has been before
00:31:46.630 coding is a major part of my life it's one of the ways I'm leaving my mark on
00:31:52.039 this world test-driven development help me move beyond the fear and anger
00:31:57.350 associated with legacy code I no longer code out of panic I code out
00:32:03.710 of love and love ultimately is what life
00:32:10.220 is all about I'm now ephemeral I'm a
00:32:15.590 software development engineer with blue boxing thank you very much
00:32:29.070 you