RSpec: The Bad Parts

Testing

Caleb Hearth

#test-driven-development

#behavior-driven-development-bdd

RSpec: The Bad Parts

by Caleb Hearth

Introduction

This video, titled "RSpec: The Bad Parts," features Caleb Hearth presenting at the Blue Ridge Ruby 2023 conference. The main focus is on the drawbacks of using certain RSpec features, particularly the practical and logical complexities that arise when adopting convenient but potentially detrimental coding practices in test suites.

Key Points

Speaker Background: Caleb Hearth has over 12 years of experience with Ruby on Rails, working in various product and consulting companies. Currently, he is involved in health tech at Buoy Software.
Common Pitfalls: The talk highlights how the use of let and subject in RSpec can lead to complex and convoluted test suites, often creating unexpected behaviors that are hard to trace.
Spiderweb of Complexity: Caleb discusses how a seemingly straightforward implementation can grow into a difficult-to-maintain web of dependencies, causing confusion for both newcomers and seasoned developers.
Real-World Example: He cites a case of a 4500-line test suite with 531 let definitions. Many were redefined or overwritten, leading to unexpected behaviors and increasing the complexity of understanding tests.
General Fixtures: The term "general fixture" is explained as creating overly complex setups that obscure functionality, which can be a major contributor to test suite complexity.
Refactoring Strategies: Caleb suggests several methods to mitigate this complexity:
- Replace let with instance variables to simplify tests.
- Refactor let and subject into standard Ruby methods, enhancing introspection yet retaining drawbacks.
- Inline variable definitions in tests, as advocated by Martin Fowler, which enhances clarity and documentation while noting that this may contradict strict interpretations of DRY principles.
Conclusion: The recommendation is to use RSpec effectively while avoiding the allure of complex utilities like let and before, in favor of simpler, more transparent coding practices.

Main Takeaways

Caleb Hearth emphasizes that while RSpec provides powerful tools for testing, over-reliance on convenience can lead to significant long-term challenges. Test suites should be written with clarity, resilience, and maintainability in mind, opting for practices that ease understanding and navigation through the code.

00:00:07.480 Hey everybody, thank you for coming. My name is Caleb Hearth. I generally prefer to keep these introductions brief at the beginning of talks, focusing on the presentation itself rather than employer promotion, since you're already here. I don't need to convince you to attend, especially in a single track conference. However, I want to mention my background as it helps underline the main point of my talk.

00:00:21.680 I've been working with Ruby on Rails for over 12 years. During that time, I've spent about five or six years each at product companies like Roku and Square, as well as consulting companies including Thoughtbot and Testable. Currently, I work at Buoy Software, a health tech company focused on building software to facilitate blood plasma donation, which includes scheduling, payments, donor management, and similar tasks. You can find my blog online where I discuss various concepts such as ethics, best practices, and testing.

00:01:02.140 You can also find me on the Fediverse at [email protected] or just by searching for Caleb Hearth; that's also my email if you need to reach out for anything else. Throughout my career, I've had the opportunity to work with dozens of Rails applications, each with its own test suites. This exposure has given me the chance to see various ways of writing tests.

00:01:26.199 Often, one little 'let' seems so easy to use and is very appealing to implement at the time, but I've observed this approach grow organically into spiderwebs of complexity. This situation becomes difficult not only for new team members to grasp but also for seasoned veterans to reason about. The 'let' method defines a memoized helper method, which executes a block only once, storing the result to return during subsequent calls.

00:01:55.440 If you explore the definition, you'll see it's almost identical to a plain Ruby method. The 'let' method can be referenced by other lets, and depending on the nesting, this reference may point to a different method across different specs. The 'subject' method can also reference 'let' methods. Speaking of 'subject', it is essentially a block with specific semantics, as there are methods like 'expected to' that utilize the subject implicitly.

00:02:28.760 You can also assign different names to subjects while maintaining the same semantics, but essentially, 'subject' serves as a named 'let'. A 'before' block is executed before each spec that it is scoped to, meaning any describe or context block containing both the spec and the before block will run that code. I will use 'before' as a stand-in for 'around' and 'after' since they're all fundamentally similar, just executed at different lifecycle stages.

00:03:07.920 'before' blocks can reference 'let' variables or any other variable within scope for a test, including 'let' and 'subject'. I'd like to ensure everyone can see this okay. I jokingly converted this presentation into a format using scripts that transformed every four lines and two characters into a single Braille character with a dot for every non-whitespace character, to address accessibility.

00:03:57.720 Now, let's look at an extreme—yet real—example illustrating why 'let' and 'subject' can be problematic. I once encountered a 4500-line unit test designed for a god object in an application. This test made heavy use of 'let', not just for the primary model but for many other method names as well. In this lengthy file, there were 531 'let' definitions across 131 method names. A staggering 78 of these are overwritten in different describe or context blocks, and some were redefined over ten times.

00:05:00.080 Among them, 87 instances define 'subject', with almost every case representing a different type of subject from the original definition. This means the model under test returned by 'subject' isn't what you expect; it's a collaborator or another sub-method being referenced instead. In this file, nearly 14% is dedicated to 'let' and 'subject' lines. Moreover, a lot of the structure supports an extremely complex general fixture, which is a primary reason tests become obscure.

00:06:06.440 The term 'general fixture' refers to when a test creates a fixture larger than necessary to verify the required functionality. In Ruby and RSpec, there are 31 'let' definitions meant to parameterize the default subject instance, which creates unnecessary setup. Most tests don't need this complexity, but due to the necessity of a couple of describe blocks, all specs evaluate this additional code unnecessarily.

00:06:38.240 The majority of these methods are structured so they are only utilized in a single spec. It would make things much simpler and clearer if they were inlined directly into the example blocks. As a result, reasoning about any given 'let' or 'subject' method is challenging due to its defined scope. It isn't uncommon for 'let' or 'subject' to reside hundreds of lines away in different block scopes from where it's being referenced.

00:07:37.800 I share this because as we refine the examples we have, I want you to recognize that real-world test suites often evolve into this complex state over time. Our aim should be to create manageable, comprehensible, and maintainable test suites that we can comfortably update. This context illustrates that when defining methods like 'let' and 'subject', it adds extra steps that complicate navigation throughout the code. Methods constructed through these macros are intentionally easy to redefine, which introduces significant challenges.

00:08:39.440 Beyond the navigation issues, these methods reference other 'let' variables across various specs, which can lead to confusion. While the code in essence seems to perform similar tasks, it may end up navigating through entirely different code paths based on which variables are evaluated. Furthermore, the behavior of 'subject' can shift across different describe or context blocks.

00:09:42.199 What are our options for refactoring? The simplest solution may be to replace 'let' with instance variables. For instance, replacing 'let :book' with '@book' maintains some perks by avoiding complexities but still has valid drawbacks. It is likely to end up being overwritten or defined in differing contexts, which decreases our ability to pinpoint which instance variable defines a given spec accurately.

00:11:03.720 An alternative approach is to change 'let' and 'subject' definitions into standard Ruby methods through memoization. While this adjustment enhances automated introspection tools, it does not diminish the risk of them being easily overridden due to the complexity surrounding these methods. The inline method refactor is a third option and is what I recommend. This technique, popularized by Martin Fowler in Refactoring, is essentially about placing the variables directly into the tests to enhance clarity.

00:12:00.640 Let’s consider how this works in practical terms. Keep in mind that although some might argue that inlining contradicts DRY principles, these principles should be viewed as a guideline rather than an absolute. Duplication of knowledge—avoiding repetition of the same logic—is what DRY aims for, not just minimizing code repetition for its own sake.

00:13:11.319 During the subsequent refinements, we should embrace the approach of duplicating setup when necessary to better document behavior. By demonstrating all required setup inline, you highlight the difficulty of your code to be easily used by exposing every collaborator and the necessary arguments to initialize any object.

00:13:43.440 Ultimately, needless complexity overly complicates specs, consequent to the invocation of overarching macros and abstractions. These practices contradict the principle I propose: aiming for resilience, resulting in clarity in scope, which would present cleaner interfaces in tests. I recommend reviewing impactful specifications within each class inline into main specs rather than trying to extract utility to lessen perceived complexity.

00:14:56.360 In summary, leverage RSpec effectively but avoid 'let', 'before', or other unnecessarily complex utilities. I've been Caleb Hearth; thank you very much for attending my talk. If you would like to discuss RSpec, Buoy, or even Dungeons and Dragons, please feel free to find me in the hallway. I extend my gratitude to my employer Buoy for sending me to the inaugural Blue Ridge Ruby conference.

00:16:06.320 We are eager to speak with you if you are a senior full-stack developer looking for a remote position where you can make a positive impact. Please come find me or visit careers.buoysoftware.com for more information. Ensure to remember that for our careers page, 'U' comes before 'O'. Thank you very much.

Blue Ridge Ruby 2023