00:00:13.920
As you probably saw yesterday, the next talk after mine was supposed to start at 3:00 PM. Now it's at 2:30 PM, so I need to cut my talk from three hours to two and a half. We might have to move a bit faster.
00:00:22.160
As you have seen in the previous talks, the theme for today is basically telling you how you should code, so please keep that mindset for the next half hour at least.
00:00:28.320
I am between you and your lunch, so I will try to make it brief and interesting enough that you forget you're hungry. However, I also have a lot of material, so I will try to go fast.
00:00:40.600
If there are any questions, please let me know. The title of my talk is 'Integration Tests Are Bogus.' My name is Piotr Szotkowski, and I work at the University. I am also the Chief Software Officer at Rebased, where we work with Ruby and Rails.
00:00:53.079
If you have any projects you'd like us to work on, please let me know. Let's start by saying what integration tests are and, most importantly, why they could be seen as problematic.
00:01:04.439
If you haven't seen this talk before, integration tests can be thought of as a self-replicating virus that threatens the health of your codebase, your sanity, and your overall life. This is a direct quote, and you should definitely watch the talk that originates from.
00:01:29.360
What types of tests are there other than integration tests? There are numerous tests with different categories and taxonomies, but let’s concentrate on three types: end-to-end tests, integration tests, and unit tests.
00:01:47.320
End-to-end tests check whether your application still works after you make any changes, integration tests check whether an object works with its actual collaborators, and unit tests ideally test a single behavior to confirm that things work as expected.
00:02:02.360
So, why would you do all of these different levels of testing? We perform end-to-end tests because we believe our tests work, though not enough to deploy to production without first clicking around in the application.
00:02:23.080
They give us peace of mind. We conduct integration tests knowing that end-to-end tests can be slow, and we want to verify whether objects work with their collaborators without mocking or stubbing them out. We use unit tests to identify what broke when something does fail.
00:02:56.120
If something breaks and you only have end-to-end tests, you typically see half of them fail, which is not an ideal way to determine what actually broke.
00:03:03.400
Unit tests also allow for outside-in development, which is an interesting technique that I personally enjoy.
00:03:16.239
Speed is not the sole reason for conducting unit tests. While they give you fast tests, which is crucial for efficient testing, their primary purpose is to isolate the object under test from its collaborators.
00:03:29.799
To achieve this, we use test doubles that act as stand-ins for collaborators, providing specified responses rather than calling the actual collaborators. This way, we can verify that the right calls were made and that our code communicated as expected.
00:04:08.159
There's a wonderful Ruby book out there titled 'Practical Object-Oriented Design in Ruby.' It emphasizes that while testing our objects with collaborators, we should adhere to command query separation.
00:04:26.440
This principle suggests that outgoing calls from our objects should either be query calls—which ask the collaborator for something without changing the state—or command calls—which instruct the other object to perform a task that does change state.
00:04:59.080
In Ruby, a method cannot be entirely devoid of return values, but you can attempt to structure your code so that state-changing methods do not return meaningful values, thus minimizing any temptation to blend state changes with meaningful returns.
00:05:22.280
There are instances where you cannot maintain command query separation, such as with a stack where you must pop off the top item. In such cases, you definitely want to change the state of the stack and know what item was removed.
00:06:01.360
However, if you can opt for command query separation, you should. According to one practitioner, when writing tests, you should stub the outgoing query calls and mock the outgoing command calls.
00:06:35.280
You can then check how your code behaves when a query call returns a specific value and verify that the command call, which is state-changing, gets executed.
00:06:54.720
For your objects, you should also test incoming calls to all the public API of that object. If the object is a collaborator of another, you will test its API from the perspective of that other object.
00:07:12.360
There’s a perspective presented by a professional who discusses the integration tests and defines them differently. He asserts that the goal of unit tests is basic correctness—essentially checking whether our object behaves correctly.
00:07:53.520
Given the myth of perfect technology, we want to be sure that our object functions correctly, assuming the rest of the system works as intended. This addresses whether the object makes the right calls, handles responses appropriately, and whether the collaborator can adequately respond.
00:08:12.960
Understanding whether the collaborator provides the right response correlates with how this unit testing can ensure correctness. Heavy use of doubles is essential for unit testing to really isolate the object from its collaborators.
00:08:32.080
If you want to do this properly, especially when using outside-in testing, you need test doubles to create an effective design, as you don’t yet have the internal implementation to rely upon.
00:08:43.920
However, there's a real danger: when your code changes, and your mocks or stubs aren't updated, they can misrepresent the object under test by showing a non-existent world state.
00:09:10.680
You can find yourself in an unfortunate situation where you have a green test suite, heavily mocked, yet the application fails to boot. This false sense of security can erode your trust in your tests.
00:09:41.920
The core problem with doubles arises from the nature of outgoing calls. If an outgoing call is defined incorrectly, or it is right but the collaborator cannot handle it as expected, this can lead to trouble.
00:10:07.360
If you encounter an issue in your tests where the mocks and stubs don't interact correctly, it highlights the symptoms of this problem. People often think that end-to-end tests will catch these mistakes, but they won't.
00:10:57.920
You cannot test all code paths using end-to-end tests alone. Thus, the need for integration tests arises. However, integration tests can often test too much to be practical and can be slow.
00:11:19.160
Also, if there are too many interactions with too many collaborators, the scenarios can grow extensively, leading to an overload of test cases.
00:11:43.200
Watch that talk for more detail; I don’t want to repeat everything here. I apologize for this bad joke about glitter and powder, but let's illustrate what I mean through an example.
00:11:54.080
Suppose we have a weather service for vehicles. Imagine I just paid tribute to someone who brought me here in their Camaro. While the weather was unfavorable for coding in Wroclaw, I can say that as you can currently see, the weather has significantly improved.
00:12:24.800
We expect the weather object to collaborate with the vehicle to ask for its location. Depending on whether it's raining or clear, we implement functionality to get the correct weather response.
00:12:46.320
Now, we implement the vehicle, but I’ll spare you the awful details for simplicity’s sake in this presentation. We ask the vehicle for its location, and if it's raining, it provides one response; if it's clear, it provides another.
00:13:10.839
Our specs pass quickly, even as a fully integrated test that tests the weather collaboration with the vehicle. But let’s assume asking for the vehicle’s location is very costly.
00:13:27.440
This integrated test passes, but it could be very slow and require much time to verify whether weather collaborates correctly with its partner, making it less efficient.
00:13:50.040
Thus, we can opt to stub it. In Ruby, the simplest way to stub something is by using 'open struct.' You can define any calls as setters, and you can receive those as outputs.
00:14:02.440
We run the stub, and again we have a fast spec that passes. However, if we want to spice up our code a bit by changing 'location' to 'whereabouts,' we create a breaking change.
00:14:29.400
Our specs still pass, even though the vehicle class can no longer respond to a request for 'location.' This shows that the integration test would reveal the problem, but only if we hadn’t stubbed it.
00:14:50.960
With verified doubles, we can test behavior as well. These ensure that the mocks and stubs check if methods exist. After removing the open struct, we can easily replace it with the Verified Doubles when running the tests.
00:15:03.199
Now when we run our specs, 'Bogus' verifies that we shouldn’t stub 'location' on a vehicle since they don't have locations—but 'whereabouts' instead.
00:15:20.680
Verified doubles ensure that the methods you’re stubbing and mocking are indeed correct and available. They provide real runtime feedback based on expected behaviors.
00:15:41.560
They allow for accurate design and drive development as they highlight what to work on next, presenting requirements you have to implement based on previous passing specifications.
00:16:02.960
Let’s also explore mocking. If you want to check whether we are making some call to another collaborator, we may be concerned with making the call rather than the return value.
00:16:20.440
In this instance, we will expect a service to receive 'weather at waro' when we have a Camaro in Wroclaw. To do this, we will mock the service call and expect to receive that input.
00:16:55.680
Expecting and spying in 'Bogus,' there's a small difference. With stubbing, you define a setup and wait for a call. When calling the service afterward, you ensure expected behavior.
00:17:12.360
As a bonus, you can easily inject dependencies into your services, which remains convenient and natural in Ruby. This approach allows for simplifying mocking processes.
00:17:44.399
Through dependency injection, the tests become manageable because they focus on behaviors and expected outcomes without tightly coupling to the specifics of implementation.
00:18:03.199
One can use verified doubles to enforce stricter contracts, thus catching instances where the mock-api using the wrong interface might lead to failure in unit tests.
00:18:14.159
The final consideration is contract tests, which remain experimental but functional in 'Bogus.' They test the actual calls and returns made by your mocks.
00:18:35.760
If we define contracts for vehicles, if we mock with certain values, we also need to ensure that our unit tests verify that those calls can succeed with the actual implementation.
00:18:46.440
For example, we can mock calls with speed values, but we need to ensure that a vehicle can handle that, and so we must run those contracts.
00:19:02.360
Mocking lets us ensure we use interactions correctly, and contract tests back this finding by validating that mocks do not misrepresent capabilities.
00:19:17.199
So, the big picture with outgoing calls focuses on ensuring correctness with verified doubles, while failing integration tests lead us to necessity for more thorough integration testing.
00:19:48.159
As I finish up my arguments here, I remind everyone that despite being in between you and your lunch, it's truly about investing time to create efficient testing frameworks.
00:20:02.439
Visit the Bogus repository on GitHub, fork it, make use of the testing methodologies, and practice integration tests.
00:20:20.440
If you'd like to dive more in-depth about practical object-oriented design in Ruby, I strongly encourage you to check out the integration test or scam talk. It's highly rated!
00:20:56.600
This content offers a wealth of knowledge regarding testing and development practices. After all, developing a robust application requires thorough and well-structured testing.
00:21:13.680
Remember, the nature of successful tests revolves around the ability to communicate well with your doubles. Don't let them veer your output away from proper unit testing.
00:21:28.160
Thank you for listening! Are there any questions?