Lightning Talk: Avoiding Sneaky Testing Antipatterns

Testing

Sarah Lima

@sarahraqueld

#testing

#test-driven-development

#rspec

#quality-assurance-qa

#flaky-tests

#refactoring

#behavior-driven-development-bdd

Lightning Talk: Avoiding Sneaky Testing Antipatterns

by Sarah Lima

In her talk at Helvetic Ruby 2024, Sarah Lima, a software developer at Toot and maintainer of Factory Bot, discusses common testing antipatterns in software development. She emphasizes that antipatterns may initially seem effective but often lead to regret due to a lack of consideration for the broader context of code behavior. The talk condenses substantial information into key points to help developers enhance their testing practices.

Key Points Discussed:
- Defining Antipatterns: Antipatterns stand in opposition to best practices and can seem beneficial initially, but often lead to undesirable outcomes later.

Positive Expectation in Tests: When writing tests, instead of expressing what should not happen (e.g., expect user.save.not_to be_false), developers should clarify what they expect to happen positively (e.g., expect user.save.to be true). It’s acceptable to quickly write tests to fix bugs, but a follow-up refactor for clarity is advisable.
Avoiding False Positives: It’s critical that tests fail for the right reasons. Developers should write tests that fail when an error occurs. Even if not employing Test-Driven Development (TDD), they should create the test to confirm failure before implementing fixes.
Use of let: While let could simplify setup, its use may obscure relationships between the test setup and the verification logic, especially as codebases grow. Sarah recommends writing test setups inline for clarity, following the Arrange-Act-Assert pattern.
Test Hooks Concerns: Using test hooks in production code can lead to discrepancies between testing and production behaviors. Instead, developers should aim for structured solutions, such as utilizing specific gems for managing dependencies.
Final Thoughts: The talk emphasizes that a good test suite should be expressive, maintainable, isolated, and reliable. Avoiding testing antipatterns is crucial for better testing outcomes. Sarah concludes by recommending further resources, including the book "Testing Rails" and the Thoughtbot blog, which focuses on testing practices.

This talk serves as a valuable guide for software developers who want to refine their testing strategies and avoid common pitfalls encountered in software testing.

00:00:05.319 Good afternoon, everybody. I'm Sarah, and I am a software developer at Toot. You can find me on my social media accounts. I am also the newest maintainer of Factory Bot. Has anyone ever heard of it? That's cool! I'm excited about just starting that.

00:00:22.480 Today, I'm going to talk about testing antipatterns. They are the opposite of best practices and may seem to work at first, but often the larger context is not considered. Down the road, you may regret some of the decisions you made in the past. So, let's dive into the first antipattern.

00:00:46.480 I'm going to try to condense this lengthy subject into a five-minute talk. Let's say you have a class `User` that implements the `save` method, and you discover a bug with it. You create a test to reproduce the bug and start by stating the obvious: you don’t want it to return false when the email is valid, for instance. You write `expect user.save.not_to be_false`, which is fine at first since you're just trying to fix the bug as soon as possible.

00:01:09.840 Alternatively, you could write that you expect it not to raise an error. However, how do you ensure that someone else doesn’t introduce a regression in some other way in the future? Instead of writing about the behavior or result you don’t want, consider writing about the behavior you do want. In this case, we want `user.save` to return true. You can write what you want directly.

00:01:40.159 Of course, if you're just trying to fix a bug, it’s fine to write it quickly as I mentioned, but then you should come back and refactor it to express the expected behavior positively.

00:02:18.239 The next antipattern is false positives. You may find code that is not behaving as expected while the test code is passing. To avoid this, ensure that you see your test fail, even if you aren’t practicing Test-Driven Development (TDD). If you aren’t writing the test first, stash your fix, write the test, see it fail, and then go back to implement your fix.

00:02:43.640 Moving on to the next point, the usage of let. I know many people don’t agree with this, but let’s discuss it. `Let` can lead to ambiguity in your tests. This occurs when the test reader cannot clearly see the cause and effect between the test setup or fixture and the verification logic because part of it is outside the test method.

00:03:07.120 This often happens with the usage of `let`. In a short test, it’s obvious, but if your application grows and you have multiple high-level lets at the beginning of a file followed by many describe blocks, it can become difficult to see which lets are relevant to your test case. You may also find that if the lets are not lazy, they can create a bunch of database records that aren’t necessary for your specific test.

00:03:41.239 I prefer writing the setup of my tests inline and only using `let` when absolutely necessary. The use case has to be clear to me. Therefore, I follow the Arrange-Act-Assert pattern, ensuring it’s all explicit inside my test block.

00:04:02.400 The next topic is test hooks. This is a straightforward example. You might be tempted to check if you are running in test mode and then require logging only if you are. This may initially help, but it can lead to writing code that behaves differently in production and test modes, which is not ideal.

00:04:36.680 Instead, you might use gems like Clearance, which can provide hooks in a more structured way without affecting your production code. So, I suggest avoiding adding test hooks to your production code.

00:05:09.639 To wrap up, I want to emphasize that we want our test suite to be expressive, maintainable, isolated, and reliable. Avoiding testing antipatterns is essential. For further reading, I recommend the "Testing Rails" book and the Thoughtbot blog, which has an entire section dedicated to testing, as well as resources on Upcase about testing antipatterns. Thank you very much.

Helvetic Ruby 2024