RailsConf 2013

The Magic Tricks of Testing

The Magic Tricks of Testing

by Sandi Metz

In the talk "The Magic Tricks of Testing" by Sandi Metz at Rails Conf 2013, the speaker explores the complexities and pitfalls of software testing. She addresses a common dilemma in the programming community, where tests often become burdensome, mismanaged, and costly rather than serving their intended purpose of ensuring code stability and reliability. Metz aims to simplify testing strategies by sharing effective methods for writing tests that are not only efficient but also maintainable.

Key points discussed include:
- Common Frustrations with Testing: Metz acknowledges that many developers express a distaste for their tests, citing reasons such as productivity loss due to slow tests and frequent failures after minor code changes. This often leads to a cycle where developers either write too few tests or none at all.

- Goals of Unit Testing: Good unit tests should be thorough, stable, fast, and minimal. This requires a clear understanding of the application’s architecture, including the identification of messages exchanged between objects.

- Types of Messages: Metz categorizes messages as incoming (queries and commands) and outgoing, explaining the significance of understanding these distinctions for effective testing. Each category has specific rules governing how they should be tested.

- Testing Strategies: Metz provides a grid for unit testing best practices that outlines when to use specific types of assertions based on the nature of messages. For example, she emphasizes that
- Incoming query messages should be tested by asserting their return values and without binding them to internal implementations.
- Command messages invoke side effects and should be tested by asserting the expected changes they cause, not just the messages sent.

- Minimizing Redundancy: By focusing on the minimal set of tests necessary for coverage, developers can avoid unnecessary complexity in their testing suites.

- Embracing Mocks: The use of mocks is discussed as a means to streamline tests, enabling rapid feedback and refactoring without the risk of cascading failures due to extensive dependencies.

- Caveats and Practical Advice: Metz underscores the importance of aligning tests with the actual responsibilities in the codebase and being cautious of over-specifying tests that can hinder code evolution.

- Final Takeaways: By applying the right strategies, developers can transform their approach to testing, making it a supportive ally in their development process instead of a hindrance. She encourages sustained focus on simplicity amidst complexity, promising that with practice, tests can become reliable tools in software development rather than a source of frustration.

00:00:07.759 Hello everyone.
00:00:15.940 If you're attending the talk about magic tricks, I have a survey for you to complete before my talk begins.
00:00:21.510 I want to ensure I don't use my allotted time for this, so please feel free to discuss among yourselves.
00:00:27.160 I’m going to ask you a question and request an answer by a show of hands. Here’s the question:
00:00:34.230 What percentage of the code that you see is tested? This question is not about your own code, so you won’t have a conflict of interest.
00:00:39.670 Please consider how well the code you see is tested. Is it 100%, 80%, 60%, 40%, 20%, or 0%? Please raise your hand to indicate.
00:00:52.330 So if the code you see is 100% tested, please raise your hand. All right, how about 80%? Now, if the code you see is 60%, 40%, 20%, or 0%, raise your hand.
00:01:05.560 For those in the 50% or less category, please raise your hand. And now, let’s begin the talk.
00:01:11.979 We are the people who embrace testing; we are the clan that drank the Kool-Aid. This is our public persona. However, you don’t have to look far to realize that many perfectly legitimate members of our community express frustration with their tests.
00:01:23.819 Many of you might say, 'I hate my tests.' Why do you hate your tests? Because they kill your productivity; they are slow. You often find yourselves on Twitter, wondering when you’ll be back from waiting for those slow tests. They hold you back. You can make a small change in your application, and while the application still works perfectly fine, all your tests might break. Now, you must go fix all those tests before you can return to actual work. It feels like such a waste of time.
00:01:51.039 If your tests are slow and break every time you change something, it’s reasonable to start questioning their value. Everyone tells you that you should have them; they probably save you just enough to make you afraid to delete them entirely. However, the cold, hard truth is that they can make you miserable. If this resonates with you, you are not alone. For many, the promise of testing has not been fulfilled.
00:02:19.390 But it doesn't have to be this way; you can have cost-effective tests. Good tests aren’t magic; they are magic tricks, and since they are tricks, you can learn them. There are a few simple and straightforward tricks I’ll explain in the next 28 minutes. For many of you, this will involve reconsidering the number of tests you wrote or previously oversaw.
00:02:38.130 Often, we find ourselves writing too many tests or testing the wrong things. This causes frustration and dissatisfaction with the testing process. Today's talk will specifically focus on unit tests; the magic tricks apply to unit tests, not integration tests. Integration tests can be complex and difficult to manage.
00:03:09.519 In integration testing, you poke one side of your application, and a series of messages and changes flow through various objects before reaching the end. In contrast, unit tests narrow the focus down to a single object, allowing for precise validations of behavior. They help confirm that every individual piece behaves as expected.
00:03:40.780 To summarize, unit tests focus on the internal functionality of one object at a time. We want our unit tests to be thorough; we want them to logically prove that the object behaves correctly. This means we want them to remain stable, so that changes to implementation details don’t break the tests.
00:04:02.709 Additionally, we desire speed. Slow tests can hinder development, and maintaining the parsimony in our test code will allow us to focus on what matters. So, we seek thorough, stable, fast, and few unit tests.
00:04:28.110 Achieving this requires clarity of vision about your application. I invite you to do a thought experiment: close your eyes, if you need to. Imagine the application you're currently working on and visualize it running in memory; picture the objects and the messages passing between them.
00:04:51.420 For many of us, it looks like a complicated tangle of spaghetti code. When our applications appear this way, it’s no wonder that maintaining tests becomes a challenge as well. The way out of this thicket is through focusing on messages and employing the magic tricks I will share with you.
00:05:07.060 The objects we will be testing have a very straightforward perspective regarding messages. Each object operates as a black box, managing an internal state while only exposing a specific interface for communication.
00:05:27.060 An object knows about messages from three places: incoming messages from other objects, outgoing messages it sends to others, and messages it sends to itself. Incoming messages can breach the containment of the object, so the internal workings must be protected.
00:05:46.330 Outgoing messages must also uphold the same level of protection, making sure that the inside of an object does not get exposed. Self-sent messages are contained within the object's own boundaries, as they do not expose anything externally.
00:06:02.290 These messages come in two categories: queries and commands. A query message runs without side effects, while a command message has an effect but does not return a directly useful value. It's important to distinguish between the two types of messages for effective testing.
00:06:22.060 Some of you might have encountered situations where a message inadvertently acts as both a query and a command, leading to unforeseen consequences. This can create dependencies in your code, making it crucial to identify what type of message you are dealing with.
00:06:45.620 In our testing endeavors, we will now look at a grid that classifies messages into six different cells, each with its own testing rules. My goal is to walk you through how to populate this grid effectively.
00:07:04.620 We'll start with incoming query messages. For example, consider a bicycle class with a diameter method. This method is a query; it calculates and returns a result without causing any state changes. To test it, we will create a new instance and verify the returned value.
00:07:22.120 This assertion checks if the output is within a specified margin of error. It’s a direct comparison of results without concerning the internal operation of the method—resulting in a straightforward test.
00:07:47.760 Continuing with tests for incoming query messages, let’s consider a gear class that implements a gear_in_inches method. This method sends a private message to itself and another to a collaborating object, effectively returning a value while ensuring no side effects occur.
00:08:04.810 The approach to testing remains the same: we send the message and assert the value returned. This means testing the interface rather than focusing on the implementation details, allowing for safe refactoring later.
00:08:25.390 Now let’s discuss incoming command messages, such as a set_cog method in our gear class. This method takes an argument and modifies internal state, potentially returning a result. In this instance, we should test the command aspect of this method.
00:08:40.580 To test it, we’d send the message that triggers a change and follow that up with an assertion about the side effect caused. This indicates that incoming command messages should be tested by asserting the expected immediate outcomes.
00:09:04.160 Now we move into a crucial aspect: do not test private methods directly. If a public interface correctly relies on private methods, testing them only creates redundancies and can lead to fragility in your tests.
00:09:26.470 Instead of having tests that bind you to the current implementation, focus on what the public interface promises. Keep this in mind to allow for safer and easier refactoring of your code.
00:09:50.090 As we proceed with the second pattern, remember the core guideline: we should avoid making expectations on private messages unless absolutely necessary. It's essential to balance clarity and testing effectiveness.
00:10:10.030 Next, we’ll observe outgoing query messages—similar in testing rules to incoming query messages. An outgoing query, like sending a diameter message to another class, is also devoid of side effects and should be treated accordingly.
00:10:31.550 Testing the outgoing query message with assertions about the result is redundant; the incoming message does all the necessary work. So, we should avoid verifying the outgoing queries per the previously stated rules.
00:10:49.340 For outgoing command messages, let's say we have a set_cog method that not only modifies internal state but also sends a change message to an observer. The observer notices this change in real time and reacts accordingly.
00:11:22.230 This raises the question of testing: should we test the side effects created by this outgoing command message? Reach out to the observer message; this could lead to unnecessary integration tests that would bind us to distant side effects, compromising our unit tests.
00:11:43.890 Thus, instead, we should use mocks here to verify that calls to the relevant methods happen without needing to check what lies beyond. Ensuring that the closest edge is validated while observing stability in tests.
00:12:05.820 Mocking allows us to maintain that bounds of testing where we only assert the message sending, not the distant side effects. This practice should help keep our unit tests focused, fast, and reliable.
00:12:28.230 To summarize, testing rules for outgoing command messages state that we should use mocks for expectations, avoiding unnecessary bindings with intermediate objects when it can be easily avoided.
00:12:50.160 In conclusion, I encourage you to adopt these principles in your testing processes, aiming for simple, effective tests focusing on the essential aspects of your code, tailoring them to meet your team's needs as best as possible.
00:13:15.240 As the talk draws to a close, I want to thank you all for your time. Please keep exploring these magic tricks and remember that good testing can transform your development experience. Test quality can lead to an overall improvement in productivity and job satisfaction.
00:13:43.890 Ultimately, with practice and persistence, you can make testing your ally rather than a burden. Thank you!