00:00:15.440
Thank you everyone for coming here. As he said, my name is Ryan Davis. I'm no stranger to the world of Ruby; I'm a founding member of the CLR Be, the first and oldest Ruby aid in the world. Currently, I'm an independent consultant based in Seattle and I'm available for work.
00:00:30.810
I'm also the author of many tests, which happens to be the most popular test framework in Ruby. I mention this because I’m somewhat astounded that the r-spec is also quite popular, apparently ranking high.
00:00:38.190
Toby's talk earlier was amazing; in just 49 slides he managed to describe the physics of the solar system, which is quite mind-blowing. I feel like I've missed the mark here, only describing a test framework with 30 to 32 slides. So, let’s segue into setting expectations. This talk will be code-heavy; the focus will be on the what, why, and how of writing a test framework.
00:01:06.299
There are 337 slides, which amounts to about 9.5 slides per minute—fifty percent more than I’ve ever done before. That means I’ll be talking quickly, and I encourage you to open an editor if you want to follow along, as I was informed that many attendees got halfway through last time and then lost me. However, do not worry if you fall behind; you can download the GitHub repository and follow along at your own pace.
00:01:39.030
This talk is an adaptation from a chapter of a book I’m writing on many tests. By the end of the session, you should have a solid grasp of how to write your own basic test framework from scratch, focusing on assertions, which are the atomic unit of any testing framework.
00:02:22.900
The famous quote not actually attributed to Benjamin Franklin states: "Tell me and I forget, teach me and I may remember, involve me and I learn." The exact origins of this quote are somewhat murky; it has been attributed to many, but the essence highlights a significant issue with code walkthrough talks. While some code walkthroughs are beneficial for understanding implementation, they can often lead to a passive learning experience where you might remember less than you hoped.
00:03:02.790
The real challenge is when the focus shifts solely to the implementation itself—what each line of code does—resulting in a dry and forgettable experience. With that in mind, I won't be merely discussing a specific implementation, but will instead demonstrate the principles of building a test framework from the ground up, using assertions as our starting point.
00:03:42.970
Neil Gaiman's famous quote could be paraphrased as: "For storytelling, begin with a simple assertion." So let’s start with the simplest form of an assertion, which basically takes a single expression and evaluates its truth. If the evaluation fails, the assertion throws an exception. To begin with, here is the plain old assert in its simplest form.
00:04:22.460
The assert function will evaluate to true if the expression passes; it fails on anything that does not. At this point, all you really need to know is how to create an assertion. There are various approaches to asserting values; for example, you could choose to raise an exception or simply collect results—and while each method has its trade-offs, the critical aspect is to report a failed assertion at any point.
00:05:04.639
I opted to raise an exception because it interrupts execution and immediately draws attention to the error. Now if we run the expression with a true value, the assert function would handle it just fine. However, when tested with a false value, it raises an exception and halts the test suite until resolved.
00:05:41.080
It’s important to note that this failure reporting currently indicates where in the assert the raise occurred; however, for practical clarity, we want it to reflect where in the actual test the failure originated. The stack trace must lead back to the line of code where the assertion was made, rather than where the assertion function was called.
00:06:22.270
We can clean this up by altering our exception handling code. By using Ruby's built-in caller function, we can specify that we want the backtrace to show the relevant lines where the failure occurred, ideally enhancing the developer's ability to debug effectively.
00:06:49.055
With the plain old assert mechanism established, let’s now add a second assertion function. The next most required assertion for my tests is to check for equality, a requirement in at least ninety percent of my tests. The implementation is remarkably straightforward: simply pass the result of a equals b to assert.
00:07:36.880
However, while the implementation passes functionality tests, the error message for failed assertions needs improvement. The error message references the assert equal in the backtrace, which, while useful, lacks immediate clarity if it doesn't point back to the specific line that failed.
00:08:29.780
To enhance clarity further, let’s allow for optional error messages when assertions fail. This adjustment will lead to clearer output that identifies exactly what went wrong. When it comes to floats, a critical lesson I wish to emphasize is: never test floats for equality.
00:09:20.890
Instead of using assert equal as you would for integers, we will develop a new assertion solely for comparing floats that checks whether the difference between the two numbers is within an acceptable margin of precision, which we're defining as 0.001 for now.
00:10:03.420
With this newly created assertion function, we now have a generalized assert function that can cater to various assertion types, including our special float assertions.
00:10:37.240
From here, it would be beneficial for you, after the conference, to think about your favorite items to test and consider how you would write assertions for them. Writing your first test should encourage you to write many tests, prompting you to separate them in the name of organization, refactoring, reuse, and clarity.
00:11:39.770
Methods offer a straightforward approach to separating individual tests from one another, allowing for greater organization and clarity in their execution. By simply defining a test method that accepts a descriptor and a block of code, we can ensure that the tests stay organized and visible.
00:12:30.640
However, we run into difficulties with shared variables when using local variables, as they can lead to interdependencies that cause one test's modification to affect the outcome of another, ultimately violating essential testing principles.
00:13:28.020
To resolve this, we can encapsulate each test in its own method, thus independent variables can be created within each scope without affecting others. Ruby provides a simple way to do this, establishing three methods, each with their own localized variables that won’t interfere with each other.
00:14:32.440
Methods don't add any overhead costs; instead, they reduce hidden dependencies and complexities in your tests. Once we have these separate tests wrapped in methods, we should also ensure that they are executed within their own context when called.
00:15:22.700
Next, wrapping methods within classes allows us to organize our tests even better. However, we must address how to invoke these test methods after wrapping them in a class structure. We can take the responsibility for running each test method out of the user’s hands and instead allocate that to the class instance itself.
00:16:21.780
This means, while we instantiate a class for our tests, we can also build an instance method that adds responsibility for running individual tests to the class instance for easier management.
00:17:18.330
It's efficient to have the class itself manage its tests rather than relying on external externalization. Therefore, we can utilize instance methods to handle the execution of test methods systematically, ensuring that they run themselves.
00:18:11.850
Further simplification of running tests can be accomplished by using public instance methods that contain the tests and filtering them accordingly, thus creating a method in our class that can dynamically collect all public test methods.
00:19:00.450
In order to ensure that our tests maintain DRY (Don't Repeat Yourself) principles, we should consider subclassing for shared test behaviors. This leads us to create a central test class to inherit from, thus allowing all our test classes to benefit from code reuse.
00:19:50.000
By subclassing a common ancestor, we also streamline our organization, ensuring that each test suite is easily accessible and runnable. Once we establish a common test class, managing the execution of tests through the class itself enhances usability.
00:20:45.460
Additionally, we’ll automate the execution of all tests, using Ruby's built-in class-inherited hook to ensure that new subclasses automatically register their tests. This creates a dynamic workflow of initiating these tests.
00:21:32.120
At this stage, while we can initiate tests quite effectively, we could enhance the framework to report outcomes effectively as well; the goal now is to give feedback on what the results of running tests were.
00:22:28.040
To improve the reporting process, we can print a dot for each test run, giving immediate feedback on how many tests have executed, and this enhances user understanding of the test progress.
00:23:20.060
Now, when tests fail, they should not halt the entire process nor lead to convoluted backtraces. Instead, we will allow all tests to run. Additionally, we will implement a mechanism for cleaner output while catching exceptions.
00:24:15.360
We will refactor the reporting system to better handle output while separating test logic from output handling, which will lead to cleaner and more maintainable code.
00:25:08.070
To do this, we can extract reporting logic into its own class while preserving functionality within the test classes. This will cleanly segregate responsibilities for better maintainability and understanding.
00:25:55.830
In simplifying our output, we can keep track of any failures without mixing them with successful test reporting, creating a clearer summary in the end for better final results.
00:26:43.810
Once we've streamlined the structure of our reporting and failure tracking processes, we can take additional steps to further clarify the output by renaming key functions to better reflect their new responsibilities.
00:27:39.060
Incorporating randomization into our test execution can assist in identifying tests that may depend on previous test states, thus ensuring that our tests remain independent and continuously check for hidden dependencies.
00:28:39.300
With all those pieces in place, we end up with a robust yet straightforward testing framework that builds upon simplicity and principled design.
00:29:27.830
While we've accomplished a lot in terms of functionality and usability, the final array of features is essentially a foundation that can be built upon, leading to further enhancements down the line.
00:30:08.850
With that said, the project of micro test offers a swift, clean, and effective alternative to broader testing frameworks, ushering in not only speed but also maintainability.
00:30:52.680
As we conclude, I want to share that there's always room for improvements and adaptations. The feedback cycle, the changes, and performance enhancements I touched on today may lead to even more refined approaches in future iterations.
00:31:47.780
As I move to wrap up my talk, keep this in mind: the journey into testing frameworks— even simple ones— begins with understanding core principles of testing, which allows for informed decisions on how best to move forward.
00:32:35.590
Thank you again for attending. I’m also looking forward to feedback on my book draft that's forthcoming and I'm optimistic about the response to come. Please follow me on Twitter for announcements, and don't hesitate to reach out with further questions.
00:34:02.760
I am now available for any remaining questions you may have.
00:34:18.620
To address the question on whether whiskey is required for using MiniTest, the answer is a definitive no; I only started drinking recently.
00:34:32.630
I would encourage using micro tests in team collaborations; understanding core frameworks demystifies tools.
00:34:42.070
Comparing MicroTest to MiniTest, both share significant functionality, but MicroTest is clearer and simpler in execution.
00:35:09.660
Moving ahead, I plan that MiniTest can adapt features to make it more efficient. Adding an abstract reporter and refining plugins could be beneficial.
00:35:53.360
Ultimately, becoming acquainted with which additional features provide context-specific enhancements and user-friendly adjustments will be crucial as time moves forward.
00:36:27.950
Overall, I welcome any additional questions and I'm eager to assist wherever possible.
00:36:51.510
Thank you once more for your time, and I hope you learned something valuable today.
