00:00:10.410
Hi everyone, and welcome to my talk entitled 'How I Learned to Stop Worrying and Love Unit Testing.' I'm Valerie Woolard Srinivasan, a software engineer at Panoply, where I help build tools for your favorite podcasts.
00:00:15.730
You can find me in the hallway later or on Twitter; my handle is @valeriecodes for podcast recommendations. But right now, we're going to talk a little bit about unit testing.
00:00:28.029
Before we dive into the talk, I want to take a moment to appreciate our location here in the beautiful city of New Orleans. I also want to borrow an Australian tradition called the acknowledgement of country, which I first saw Pat Allen do at Nation Ruby. This is a way to acknowledge the native people who first lived on this land. I would like to take the time to acknowledge the Choctaw, Houma, and other tribes, the traditional custodians of this land, and I extend that respect to other indigenous people who are present. Thank you for taking the time to appreciate their contributions to this land and this culture with me, and for coming to attend this talk.
00:01:10.030
Let’s get started. I'm happy to be kicking off the testing track. I hope that this can teach you something if you're already doing testing and give you an idea of where to start if that's something that's new to you.
00:01:21.009
I chose this GIF of Kermit the Frog partly because I love Kermit the Frog and it made me laugh, but the other reason is that I love the carefree nature of Kermit. I think a lot of the power that testing gives us is confidence. Good tests allow you to be confident that you won't accidentally break something.
00:01:30.880
Good tests allow you to verify that a change in your code isn't actually changing the functionality of your program. They let you worry less. Maybe they don't let you be as carefree as Kermit, but I'm hoping they can come close. So let's start right off the bat with what our tests are even for in the first place.
00:02:05.229
Some of you may be fairly new to software and just know that testing is probably something that you should do without really knowing why. I talked a little bit about confidence; one way you can think about tests is as an act of kindness. When you write a test, you're taking the time to verify that your code works. This is a favor to your teammates and your future self. Think of it as an investment.
00:02:37.739
Coming into a new codebase or starting on a new project often feels familiar in certain ways. Maybe it's a language you're familiar with or a kind of program that you've written before, but there may be little gaps or assumptions that you make that are incorrect. It's kind of like playing croquet with a flamingo and a hedgehog. There are ways in which things might seem familiar, but there are ways that they are really not at all what you expect. Your old assumptions may not hold true, and you have to figure out where the gaps are between your understanding and reality. Tests can help you bridge that gap.
00:03:14.961
Who are tests for? Let's take a moment to talk about that. There's a bit of a story behind this quote. My parents are both in the art world and got me this book written by a friend of theirs about being a professional artist. I'm pretty sure they were hoping I’d be one, but I'm not. I really love this book; it's called 'Art and Fear.' I think it's incredibly relevant to programming and all sorts of creative work.
00:03:25.379
I'm really obsessed with it. This will not be the last time I quote it in this talk. I have an idea for this talk where I just lecture about this book in the context of programming, but that's for another day. I had to sneak it in somehow. So, to all viewers but yourself, what matters is the product, the finished artwork, to you and you alone. What matters is the process.
00:03:43.480
In the context of software, the process of creating your artwork is often a collaborative one, so it's definitely a little bit different from being a solo artist unless you're just working on a side project by yourself. That said, writing tests is part of the process of writing software, not the finished product. A user of your application doesn't care about your tests as long as the product works. You should care about writing tests because they will help you build a product that works.
00:04:16.239
If you're a user, you’d probably rather use a product that works perfectly and has no tests, but as the software developer, you'd much rather use a product that has some issues but is well tested. So the first and perhaps most obvious function of tests is protecting against bugs, or at least catching those bugs before they make their way to production.
00:04:43.979
The software that you're working on is likely to be a very complex system. You're probably not going to be the only one modifying it, and it's not going to be possible to keep track of or understand what's going on in every part of the application at any given time. A test, therefore, can serve as an audit trail of how something is supposed to work or used to work. When a test fails, you should have some proof or paper trail of when that test used to work so that you have an easier time identifying what broke it.
00:05:03.780
You can use tests to prove that your code does what it says it does when you write it. Ideally, you can use a continuous integration process that will test your code every time you make a new commit. That way, when your test is green on one commit and red on the next commit, you can be pretty confident that some code you changed in between has broken it. Good tests, coupled with continuous integration, will allow you to pinpoint and quickly correct any code that breaks those tests.
00:05:49.710
Another great asset that tests can provide for you is documentation. You may be more familiar with writing documentation in the form of comments or internal wikis, but that can get out of date very quickly and cause you some trouble. For example, here I've written a method that always returns the number four and I've written a comment talking about my method that only returns the number four.
00:06:21.650
However, let's say I change this method so it now returns the number five. There is nothing that is going to force me to update that comment to reflect the actual state of the codebase. If I change the return value of the method, the comment remains there as a terrible blatant lie. If you’re using continuous integration, tests must be updated when they fail, so they're more likely to reflect the current state of the codebase than your comments.
00:07:09.389
Take, as a counterexample, this test that I've written for this same method that always returns the number four. In the above example, nothing forces the comment to be updated when the output of the function is switched to five. In a less trivial case, this could lead to comments giving future developers, including yourself, misleading information about the actual state of your code.
00:07:42.650
Another good one is the comments that say, 'Talk to so-and-so before you change this,' and so-and-so hasn’t worked for the company for six months. Good tests can serve as an introduction to your code. Well-written tests, along with well-named functions and variables, should explain the desired behavior of your code as well as testing its functionality.
00:08:20.990
Tests can also help prompt you when your code is getting too complex and encourage good practices. As Sandi Metz said in 'Practical Object-Oriented Design in Ruby,' tests are the canary in the coal mine. When the design is bad, testing is hard. Writing tests as you write your code will help give you an idea of where the complexities in your code are.
00:08:51.490
If you're not sure how to test a method, that method might be too complex and need to be broken up into smaller functionalities. If you find yourself having to write lots of scaffolding in order to even run your tests, that can signal that you may have too many external dependencies. The Law of Demeter in object-oriented programming states that each unit should only have limited knowledge of other units and that a unit should only talk to its immediate friends.
00:09:23.459
So, unit testing is a great way to enforce that methodology. How much does your class actually need from its neighboring classes? If it's too much, then writing unit tests will be more complicated, which should serve as a hint to you to go back and simplify the code that you're testing. Another thing that tests do is allow for confidence in refactoring.
00:09:54.270
If you've got some code that's written in a way that makes it very difficult to reason about, you can't rewrite it. If you can't reason about it in the first place, how are you going to preserve all that functionality without knowing quite what all that functionality is? Without tests, it's impossible to make changes to your code and ensure that you haven't broken other things.
00:10:20.130
You are indeed living very dangerously because your only conception of how your code is supposed to work is within your own head, and you have no way of verifying that this actually lines up with how it works or how it was originally intended to work, which could be three completely separate things. Efficient refactoring is only possible with a well-written test suite.
00:10:51.290
This is an example of how taking the time to write good tests will save you time in the future. Refactoring is a hugely important part of writing good code, just as revision is an important part of writing. Remember in high school when you were writing an essay? You would type and type until you reached the word count, and then you would print it out and never look at it again.
00:11:06.740
That's not the way you want to write code. You're not only depriving yourself of the chance to introspect a bit more about the code you've written while it's fresh in your mind, you're depriving future developers of more insight into how your code is supposed to work and how to tell if it's working. This knowledge is essential for making changes later.
00:11:42.110
Here's 'Art and Fear' again. This quote struck me in the context of tests because I feel like it's also kind of about refactoring. You make good work by, among other things, making lots of work that isn't very good and gradually weeding out the parts that aren't good. I think this is absolutely true about programming as well.
00:12:06.440
I do most of my work by just putting my fingers to the keyboard and reasoning through the problem at hand in whatever way first comes to me. This is often pretty repetitive and sometimes needlessly complicated, but this rough draft and starting point is essential to growth. Being able to edit and make improvements to your code over time is an essential part of improving as a developer.
00:12:48.740
But you can't remove the cruft without understanding exactly how your code is supposed to work and, crucially, when it stops working. That's the insight testing gives you. When you write tests, it gives you a chance to look through your code to get a sense of how you feel about it.
00:13:03.389
Are the variables well named? Is the logic clear? Do you feel like you're doing something hacky? Do a gut check and make changes where you see fit. After all, you've got some tests now, so that shouldn't feel quite as scary. So these are the things that tests can do for you, but what exactly makes a good test?
00:14:02.580
Obviously, not all tests are created equal, but what differentiates them? A good test suite really only has to do two things: it needs to break when code that you've changed has broken your app and not break the rest of the time. Both of these things turn out to be much easier said than done.
00:14:30.949
As it turns out, it's very difficult to predict what parts of an application are most likely to break with future changes and write tests in a way that exposes those things. It's also quite easy to write tests that break in ways that don't actually indicate failures in your code, like a copy change or a timeout, because something takes too long or the change in formatting of an output.
00:15:14.759
You should also be wary of tests that might be prone to timeout or assume anything that could change, such as the year or the environment in which it's being run. This talk is about unit testing, but we haven't really differentiated the different types of tests and what they really mean.
00:15:42.279
You'll hear slightly different definitions from different sources. These are the definitions that are used in the Ruby on Rails guides: there are unit tests, functional tests, integration tests, and system tests. A unit test tests the smallest functional unit of code that it can, such as a single method on a single class.
00:16:03.420
An integration test tests aspects of a particular workflow, and thus tests multiple modules and units of your software at once. You might create an integration test to ensure that users can log in or create accounts. Functional tests look at controller logic and interactions. They are testing that your application handles and responds to requests correctly. This is for example testing your controllers in a Rails app.
00:16:43.020
System tests allow testing of user interactions with your application running tests in either a real or headless browser. Again, in the case of Rails, system tests are like an automated QA script and probably most closely mirror the way you would perform manual QA in an automated environment.
00:17:03.360
So out of all these types of tests, I'm choosing to focus on unit tests. Why is that? Of the test types I talked about, you'll notice that unit tests are by far the simplest. I like them because they're easy to write, easy to run, easy to reason about, and they also help encourage modularity and cleanliness in your code.
00:17:41.829
As we discussed earlier, individual methods are probably the things in your code that you have the best understanding of, making them the easiest to write good tests for. If your code is clean and modular, well-tested units should lead to a functional application. Unit tests are also very fast to run since they have the fewest dependencies and don't require spinning up something like a headless browser.
00:18:04.519
That said, there will be times when you have to write other types of tests, but you should be thoughtful about when those times are. Because of the overhead involved, it's probably a good idea to have integration tests for the most critical workflows of your app, for example, in the case of a company, the things that would be most likely to lose you a lot of money quickly if they were broken and shipped to production.
00:18:31.093
System tests can be used to test important workflows, but can be slow and brittle, so they should be used with caution to supplement a robust unit test suite. Here are some of the things that you might be interested in testing. This is by no means an exhaustive list, and there's plenty of documentation to be found online about how to test different things.
00:19:03.090
But these are some of the things I find myself testing most often. The most important thing to keep in mind as you start testing is to keep things simple. Each test should look at only one very small thing. Each of the bullet points I've listed here are examples of assertions, establishing the idea of what your code should do.
00:19:32.570
When you run the test, the computer will tell you if it actually does or not. I've added an example of using RSpec to run a few tests in just a simple IRB console. It may be a little too small to read, but these slides will be online if you want to take a look.
00:19:53.359
It's basically saying expect 2 plus 2 to equal 4; it returns true. I then expect 2 plus 2 to equal 5, and I get an error: 'Expectation not met error. Expected 5, got 4.' So that's kind of the syntax you'll be looking at.
00:20:10.449
The first time I run the test, it passes; when the test fails, the failure message includes the expectation that was not met, the expected value, and the actual value. It's important, as you start testing, to glean as much information as you can from these error messages because they can tell you a lot.
00:20:41.200
Don't just say, 'Oh, my test failed,' go back to the drawing board. Really take a moment to think about what it's trying to tell you. You can test the exact value of a return value. Incidentally, you can also use array matching, greater than, less than, includes, the whole deal. You can check that a method causes another method to be called, check that a job is queued, check that a database object is created or destroyed, or see if something is true or false.
00:21:09.670
This is, as I mentioned, not an exhaustive list, but instead is meant to get you thinking about how you might start writing unit tests for your code. So what is a unit test? As we noted, a unit test looks at the smallest possible unit of code. In the case of Ruby, that's a single method on a single class.
00:21:38.920
In unit testing a method, you should think about all the reasonable inputs to that function as well as how the method should respond to invalid inputs. If your method uses branching or conditional logic, you should have a unit test that hits each possible branch or combination of branches.
00:22:14.350
Here's an example of how I might approach unit testing this method that I wrote called 'greet.' It's a fairly trivial method. It takes someone's name and says hello to them. If it gets a weird input, like anything that's not a string, it just says hello and mentions that it didn't catch the person's name.
00:22:56.840
That's a design decision on my part. I could also throw an exception or just call to-string on any input that I received. The code examples here use RSpec, but the general ideas should be applicable to whatever testing framework you're using.
00:23:28.710
Because I have two possible conditions in my return values, as I mentioned, I want to at the very least test every branch of any conditional logic that I'm using. So I know that I need at least two unit tests. If I wanted to be especially thorough, I might test for other edge cases like different types of non-string input. You can see how, if you have many branches in logic, this can multiply and get complicated very quickly.
00:24:06.870
So let that serve as yet another incentive not to nest too many conditions in a single method. I've written simple tests for each of the possible conditions; in this case, I've written two unit tests. One tests the method using the expected input—this would be called the happy path, where I call the method using my name and it responds with 'Hi, Valerie.'
00:24:45.430
Then I call the method using the number two, and I make sure that I get the response saying, 'Hi, I didn’t catch your name.' Just as a syntactical note here, I apologize for having to use a ternary operator. I did it so I could fit everything on the slide. If folks are not familiar with that syntax, the basic idea is that it’s just an if-else statement.
00:25:07.000
The if part is behind the question mark, and the return value for the if follows right after that, and then the return value for the else condition after the colon. The site Better Specs has tons of resources on testing best practices and also ways to write your tests down to the grammar of how they should be written.
00:25:26.380
This ensures that the printout of what failed and passed in your test suite is easiest for you to read. The same principles of readability that apply to your code are probably even more important in your tests. My general rule is that an English speaker who doesn't know Ruby should be able to read your test and have an idea of what it's doing.
00:26:08.610
In RSpec, things are named very deliberately to allow for this. You'll notice that the test syntax includes 'it' and then a string, 'do,' and in the string, you can give an English description for what you're actually testing for. The syntax here, 'expect this thing to equal this thing,' should be fairly easy to parse.
00:26:50.860
The less readable your code is, the more straightforward your test should be, although both should ideally be as straightforward as possible. So we’ve talked about what’s good about tests and why you should write them, but we’ve probably all been in or will be in situations where writing tests is skipped or overlooked. So why is that? What makes testing hard or causes it to be passed over?
00:27:22.960
A lot of the challenges around testing boil down to time: developer time, computational time, and the passage of time. Writing tests takes time. The first time you're writing something, you're probably testing all your code manually in a development environment. You've convinced yourself pretty well that it worked, and it doesn't seem worth it to write tests.
00:27:55.970
Then why write automated tests? That takes time you could be spending writing your next feature. Tests also take time to run, and if you've configured a continuous deployment environment where your tests have to all pass before you can deploy to production, this is, in one way, very deliberately slowing down your deploy process.
00:28:25.610
This can make it more frustrating to push, say, an urgent fix through, especially if those tests take a long time to run or have spurious failures. You can mitigate this by writing fast tests, such as unit tests, keeping those tests simple, and using tools like Zeus to help load your tests faster.
00:28:50.310
Testing anything that involves time can also be challenging. Let's say you want to check that a timestamp is correctly recorded, but fractions of a second elapse between the time the object is saved and the test is run, making the matching test actually fail. What if your test server is in a different time zone than your development and production environments?
00:29:23.960
These issues can be mitigated by using gems like Timecop, which allow you to freeze time in your testing environment. The final time aspect of testing that can be difficult is knowing at what point in the process to write your tests.
00:29:47.920
If you're waiting to write them until the end or trying to write them long after writing the code under test, it can be hard to remember what exactly you were doing and what things are most important to test. We tend to overestimate our abilities to remember things, and you're likely to forget the context for your code and decisions very soon after writing it.
00:30:17.920
I think we've all probably experienced the moment of coming to a piece of code that we wrote maybe six months ago and not remembering anything about it or the state of mind we were in when we wrote it. Even if you're not using true test-driven development, you should be writing tests alongside your code.
00:30:41.050
Writing out an idea of what you want your tests to look like before you begin writing code can be a helpful exercise in thinking about how you want to structure your code. Another really challenging part of testing is if you’re faced with an app or codebase with no tests and trying to figure out where to even start testing it.
00:31:06.270
Instead of trying to take on the monumental task of writing tons of tests at once, make sure that every new piece of code that you add is well tested. You can choose a testing framework and use tools to get an idea of your test coverage and then start chipping away at it.
00:31:32.050
I like to think about the Boy Scout rule: leave your campsite cleaner than you found it. Leave the code cleaner and better tested than you found it with every pull request. Are you fixing a bug? Stop before you fix it, write a test that exposes it, and fails. Now go fix the code and turn it green.
00:32:07.110
Are you writing a method? Make sure you have a separate test for every branch of its conditionals. Think about edge cases. Let's say the method gets passed a nil value, a string instead of a number, a negative number, zero, a really big number. Think about how you want that method to respond in those conditions and write tests that validate that behavior.
00:32:30.340
Favor tests over comments as a means of explaining your code to future engineers. Get everyone on your team to agree that testing is important and agree on a strategy. You can use tools like Coveralls to give you feedback about what code is and isn’t covered by your existing test suite.
00:32:52.180
Once you have a starting point, you can set a goal, for example, to increase code coverage by 5%. You can also decide that all pull requests need to include tests in order to be merged.
00:33:06.750
Now, another thing that's really hard to test is external systems or dependencies. Let's say that your code makes a call to an external API and then parses that response. It's inefficient and clunky to actually make that call every time you run your tests.
00:33:33.040
So what can you do? You have a few options: you can mock out a valid response in your tests and just make sure that you're doing the correct operations on it. You can use a tool like WebMock to construct fake HTTP responses or a tool like VCR to make a real request once and record the result, performing all future tests from the recording.
00:34:01.740
Keep in mind that both of these methods rely on the current state or your interpretation of the current state of the API response format, and if that changes, it has the potential to break your live app but not your tests.
00:34:27.770
With that, I hope this talk has given you some ideas for how to start testing if you haven't already. I hope it has provided you with ideas and things to think about regarding your test suite if that's something you're already doing.
00:34:49.405
Feel free to find me later or on Twitter with any questions. If you're interested in working for my company, Panoply, we’re hiring and I’d be happy to chat about that as well. Go forth and conquer!
