Quick and easy browser testing using RSpec and Rails 5.1

Capybara

Sam Phippen

17 talks

Quick and easy browser testing using RSpec and Rails 5.1

by Sam Phippen

In his talk 'Quick and Easy Browser Testing using RSpec and Rails 5.1', Sam Phippen addresses the challenges traditionally associated with full stack testing of Rails applications and introduces solutions brought by the new system tests in Rails 5.1. The presentation emphasizes the ease of setting up and executing browser tests, significantly improving the testing experience for developers.

Key Points Discussed:
- The Testing Pyramid: Phippen introduces the concept of the testing pyramid, explaining the balance necessary between isolated tests (unit tests) and integrated tests (browser testing), emphasizing that while unit tests are fast and numerous, integrated tests are critical but should be kept to a minimum due to their inherent slowness.
- Previous Challenges: The speaker reflects on the difficulties encountered in getting browser testing to work with Rails 4.2, including complications with Capybara setup, dependency requirements, and issues with testing frameworks like Poltergeist.
- Introduction of Rails 5.1 System Tests: A major focus is on the introduction of system tests in Rails 5.1 and how they streamline browser testing, making it straightforward for developers to set up tests with RSpec and Capybara.
- Live Demo: Phippen showcases a live demo where he quickly sets up a Rails application to demonstrate the simplicity of implementing system tests, highlighting that the new tools allow for immediate feedback and improved debugging capabilities, such as automatic screenshots upon test failure.
- Collaboration with Rails Community: Throughout the talk, Phippen points out how collaboration within the Rails community and contributions from various members have led to the features being discussed, encouraging attendees to engage more actively in community events.

Conclusion:
The main takeaway from Phippen's talk is that with the enhancements introduced in Rails 5.1, browser testing has become significantly more accessible and manageable. Developers can focus on writing tests without the previous overhead and pain points, thus improving their productivity and confidence in application stability. The talk encourages attendees to leverage these advancements to enhance their RSpec suites and to engage with the community for continuous learning and collaboration.

00:00:10.910 Alright, it’s not actually time to start, but I figured I’d do some vamping before we go on. Firstly, they didn’t tell me that I was going to have to follow Sara May when I agreed to speak at this conference. Usually, when Sara and I are speaking at a conference together, she is the closing keynote, and I’m right before that, so I don’t have to be the disappointing follow-up to her amazing talk. Can we just give a big round of applause for Sara and the great work she did this morning? We were literally at dinner last night, and A.B., the woman who does most of the top-level organization of this conference, was having a massive freakout because the speaker’s flight was delayed. Since I travel a lot for conferences, I was pulling up flight websites on my phone and checking out connections, thinking that it was almost impossible to make. It was pretty tense in the room, so Sara stepping in at the last moment was really huge. Thank you once again!

00:00:45.239 Hi RailsConf! It’s been a little while since I last attended. How’s everyone doing? Cool! As people filter in, please sit in the middle. I encourage those on the edges to also stay in the middle. I promise you’ll be able to see the slides. I’ve been sitting at the front middle the entire conference and I can see them. It makes the room look a lot more full if everyone sits in the middle. You’re allowed to stand up and move around. We have about 30 seconds to go, so while people are filtering in, I’m just going to get started. When I submitted this talk, titled ‘Quick and Easy Browser Testing Using RSpec and Rails 5.1,’ Rails 5.2 wasn’t anywhere close to being done, but it is now, so I have to make some adjustments.

00:01:26.869 Oh my god, is the clicker not working? Let me try unplugging it and plugging it back in. Eileen, if you could maybe bring yours up, that would be great. Oh, it looks like Keno may have crashed. No, we’re good! Thank you, Eileen. It’s why you start early so that when the clicker doesn’t work, you can keep going. Basically, since Rails 5.2 came out, I’ve had to make these adjustments, but much of the content in here is 5.1 specific. I will note adjustments on the slides that I’ve written, but for the most part, this will still tell the same story. It’s really cool when Rails versions get released around conferences so we can all talk about the upgrades and new features available in Rails. However, unfortunately, RSpec, as of today, does not officially support Rails 5.2.

00:02:17.480 But I will tell you that fixing it required me to change one internal test. Unless you have a wildly specific use case of Active Storage that almost nobody has, RSpec should work with Rails 5.2 today. This has been confirmed in several production applications, so you should be all good. It’s interesting for me to be up here because I’ve been doing this for quite a while, and Rails has changed dramatically since I started. But that’s not all that has changed; the community, the people in the room, who’s participating actively, and what’s getting committed have changed as well. Perhaps more importantly, I’ve changed a lot as well over the past several years. I’ve been speaking at Ruby Central conferences since RubyConf in San Diego in 2014, and I wanted to reflect a little on some of the memories I’ve created through those conferences.

00:04:03.510 When you give a talk, Confreaks records it and it goes up on the internet forever. Your unfortunate looks from the past get immortalized, and for me, this is great because I love this community. I’ve been working in it for such a long time, and I’ve grown up as an engineer with it, so I get these real-time snapshots of who I was over several years. Including the one time I spontaneously transformed into Justin Searles — that was not a fun experience! So, the point I’m trying to make is that if you’re new here, if this is your first time at RailsConf, this is an incredible thing that we do. It’s been one of the most life-changing experiences I’ve ever had, and I would love it if you came back next year to RubyConf in LA or RailsConf, or any time afterward.

00:05:01.740 Give a talk, participate, start a user group, go to a user group. This is an incredible community, and I’m so proud to be a part of it. So, without further ado, you all are probably in this room because you care a little bit about testing. I wanted to start by talking about the testing pyramid. This is our testing pyramid; it basically defines one model of what a well-designed test suite looks like. The idea of the testing pyramid captures two things: the first is how integrated or isolated any particular test is. When you write unit tests, perhaps a model test, those are typically more isolated — they don’t touch the entire system; they touch individual pieces of behavior.

00:06:03.150 On the opposite end, we have integrated tests. These tests spin up a browser, hit your database, maybe send emails, hit Stripe, do third-party interactions, make phone calls — all of that. They are as close to testing in the real world as you can get. The testing pyramid advises that you should have larger numbers of isolated tests or unit tests and very few integrated tests. There’s a reason for this. If we go to the top of the testing pyramid and talk about the most integrated tests, we’re likely spinning up a browser and making all kinds of requests, hitting the database, caching layers, third-party APIs, etc. Those tests are necessarily slow, just because they are doing so much work across the system. What the testing pyramid basically advises is that you should write as few integration tests as necessary to have confidence that your application is working.

00:07:10.789 Usually, you do this by testing happy paths in your integration suite and not caring too much about edge cases. That’s a little bit of a generalization, but it’s where I’m starting from. The isolated tests tend to be really fast. If I run `bundle exec rspec` in the RSpec repo itself, you can see that the tests for RSpec execute really quickly. The reason for this is that most of what the tests in RSpec do is test pure pieces of Ruby functionality on their own. They’re not going out to the file system or doing any IO. The overwhelming majority of them are really quick, so because these tests are fast, you can use them to test all of the edge cases in your system quickly and easily without worrying about the details of everything that’s going on.

00:08:24.039 You can see here that the suite executed in just under 30 seconds. So that’s our testing pyramid, and that’s a bit of high-speed testing advice.

00:08:38.260 But really, today we’re here to talk about the most integrated part of our stack: doing browser testing against our Rails applications. I’d just like to say that this used to be so awful. If anyone has ever tried to get a JavaScript Capybara setup going in a Rails 4.2 app, you'll know this pain.

00:09:02.029 For those of you that haven’t, I’d like to show you. I went away and googled ‘Rails browser testing,’ and the first result that comes up is a link to Capybara. This is where I have to pause and say: I am in no way trying to dunk on Capybara. Capybara is an excellent piece of tooling; it does what it’s supposed to do. It’s just that the documentation and guides for this specific use case are not great.

00:09:30.490 So, I go to the Capybara website, follow the gem command, add the require, and do the setup for a thing called Selenium. Selenium allows you to drive a browser programmatically from your Rails application, and Capybara knows how to integrate with it. I follow the instructions exactly, write my test, run it, and get the blow-up: ‘Capybara Selenium driver was unable to load gem.’ I’m like, cool! I slap that gem in my Gemfile, run bundle install and run my tests again. Then I get this error: ‘Please add Puma to your application.’ This isn’t going to work. This is a Rails 4.2 application; I’m using Unicorn in production — Puma wasn’t the default in Rails for a long time. You should be able to get a test response from an application without having this gem present. I honestly have no idea why Capybara is doing this. This actually made me really angry when I saw this error. I was like: this should just work, yet it doesn't!

00:11:03.910 I took a deep breath and went back to Google. The second result is this gem by Fernando called Browser, which shows browser detection. Well, that’s not really what I want; I’m trying to test with a browser, not just detect one. So I keep searching and find this article titled 'Testing Heroku in the Browser with Capybara (Rails backend),' and it’s on Medium, so it has to be accurate, right? I click through and literally the first code sample is the exact error I got. Perfect! I’m probably onto a winner.

00:11:56.200 I copy the author’s gem setup, copy the author’s Capybara setup, and run my tests. Everything passes! Easy, right? But let’s quickly look at the code sample that that author gave us and do a little investigation. Firstly, the author is passing `--ignore-ssl-errors` equals `yes` in the PhantomJS configuration. I don’t know about you, but if I'm hitting Stripe, sending emails, or doing other integrations in my tests, I’d like to have reasonable confidence that SSL is working all the way through. This seems like a really strange default to me. More importantly, this is using Poltergeist, which is a sort of headless WebKit with a JavaScript engine bolted on, which means it’s not a real browser.

00:12:53.920 I suspect most of you don't browse the web on a command line by parsing responses out of an invisible browser running somewhere on your system. You probably use Chrome or Firefox. The idea of testing correctly, to me, implies as realistic a scenario as possible. I’d much rather have a real browser if I can. So I go back to Google, scroll and scroll, and eventually get to this headless Capybara feature specs with Chrome at thoughtbot.com. Anytime I see something from Thoughtbot, I look closely because they’ve done great writing over the years to ensure we have really good resources.

00:13:34.540 I click through, and the author of the post is Derek Pryor, whom I know well. I figure this will solve my problem. He lays out all the setup you need to get Google Chrome up and running against a Rails 4.2 application. I paste this in, run my tests, and lo and behold, the browser window pops up. There’s an internal server error, but this is a fresh application with no routes, so it’s working. We’ve won, and we can now go forth and test our applications happily. However, that sure seems like way too much work to accurately test an application.

00:14:26.210 Ilene, in her keynote, made a rallying cry for us to stop doing specialized things to solve our problems. I’m not just building up to this for no reason: this problem is solved now in Rails 5.1 and 2, and RSpec version 2.7 completely addresses it. This all starts with Amanda, who gave a talk last year on building the new Rails system tests. System tests solve exactly this problem; they provide a well-tuned, Rails-optimized Capybara browser configuration that allows us to run this test easily. In fact, I don’t even need to explain it too much because I can just play this video at you, and we’ll conclude faster than it would take for you to read a blog post on how to do this.

00:15:36.840 Basically, the video shows that we’re setting up a Rails application, adding RSpec, and configuring it to create a browser test. At this point, I will say that in Rails 5.2, I’m about to make a configuration change to our Puma file that isn’t necessary anymore. In Rails 5.2, the Rails Puma config file out-of-the-box does the right thing. So we added RSpec to our Rails app, created the default RSpec install, and I changed the Puma config to properly handle worker threads. I swear that if the browser can’t boot the Puma configuration correctly, the system tests will not work. This is well documented if you’re on Rails 5.1, and it’s solved if you’re using Rails 5.2.

00:16:54.199 I should also mention that the video is not sped up at all; this is just me working in real time. Then we open our RSpec file, load our Rails helper, and here I initially type 'foo,' but that doesn’t make sense, so I decided we should call it something real, and it works. I also have the Snowman as my trailing whitespace character; I think it’s nice! Then, you add the type system to your test. This is the magic that allows RSpec to pull Rails stuff, and then you run `bundle exec rspec`. At the very bottom, a little Chrome window pops up, and I get a Capybara error instead of a passing test because I haven’t added a root to my Rails router.

00:17:58.640 Instead of the test passing, it’s failing in the browser. Rails system tests have it set up so that the exception actually gets bubbled up to you, which I think is a much better interface. So anyway, I add a controller and a route to the home index for my tests one more time, and you can see that now the tests passed. There’s a pause there while Chrome is booting, but it’s not a long one. Finally, to show you that this is real, I do the best programming thing in the world: I add a sleep statement. Then Chrome boots, and you can see it. That video sequence took less than two minutes to create, and that’s all you have to do to add system tests to RSpec in Rails 5.1.

00:19:00.670 So, the Puma configuration looks like this: the important parts are the threads set to 0, and the worker set to 0. This basically tells Puma that the test should boot up the server thread instead of one already being booted. This is important so that the Active Record test setup can do the right stuff. I obviously don't grasp it that well. Setting worker to 0 tells Puma not to boot any additional processes in the web server, ensuring processes are entirely separate from memory. This is crucial to keep our tests from breaking.

00:20:11.170 In terms of the structure of the actual test itself, all you need to do is write an RSpec describe with some description for your test, tagging it as type 'system.' I still use the hash rocket syntax; fight me on this. After that, any meaningful description is followed by normal Capybara commands like visit, expect, page to contain, tag selectors, or JavaScript evaluation, and all that. I have the sleep statement copied in because I wanted to grab a screenshot of the browser. If you don't want a full web browser and just want to assert on HTML without having to execute JavaScript, you can call this method called `driven_by` and swap Selenium out for Rack Test.

00:21:10.849 What Rack Test does is fake the communication between the real browser and a fake one that Capybara uses to pause the responses without making a web request. This is really useful for testing simple responses, as you can assert what the page contains. It’s worth mentioning that all the Capybara methods are included by default, which fails fast. One cool thing to show you is that if you use RSpec 2, and your test fails, the system testing will take a screenshot of the page when the test fails and render it in your terminal. This is an extremely useful debugging utility.

00:22:09.670 You'll also notice that it tells you the path of the screenshot, so if you need to make it bigger, you can go to `temp/screenshots/failures/rspec/example_groups/who_fails` and some random number. I’ll show you how that name gets generated in just a bit. This is an amazing debugging tool because usually when Capybara fails, it’s because it can’t see an element or some text is misaligned. Having that screenshot has proven really helpful.

00:23:09.110 So, this is great! Look how easy it was; you just have browser testing now. There’s no painful Googling or digging through blog posts, having to worry about the special artisanal browser configuration tree — it just works! This is an amazing piece of technology developed inside Rails, and I’m incredibly glad that it’s available in RSpec now.

00:23:53.230 That’s all the explanation of the feature I have. It's pretty well documented, and you can go explore further on your own. However, it wouldn’t be an RSpec talk without my diving a bit into the framework and the implementation itself. First off, I had known this was coming ahead of RailsConf last year, but I had no time to prep for it. This integration within RSpec actually started getting written at RailsConf last year when I realized that RSpec needed to support it. The pull request landed just a handful of days after the conference. It wasn’t completely done at that time, but I was really happy that I was able to get something working quickly.

00:24:49.500 The `driven_by` method is provided by Action Dispatch itself. When people test their applications with many tests, they usually write an application system test case from which they inherit, which has all their browser configuration. That implementation didn’t really make sense in RSpec, so we just added the `driven_by` method to all system tests and called it to dispatch directly to Action Dispatch to perform correctly. It then calls that method in a `before` block within your test.

00:25:49.750 If you don't specify a preferred browser, our `before` block gets evaluated and defaults to Selenium, which aligns with Rails defaults. What I’m trying to highlight is that Rails is really doing the heavy lifting here. The APIs Rails provided were clean and made it very easy for me to integrate them into RSpec. Most of the work was dealing with aspects protocols, not having issues with Rails. Let’s discuss those screenshots. The way those work is actually really interesting. The screenshot is printed to standard out as cleverly designed pixel characters, like bizarre magic.

00:26:45.950 Just like we added a `before` hook to your tests, we also add an `after` hook. What we do is grab standard out and rewrite it to be a StringIO. For those unfamiliar, StringIO is an object you can write to that builds up an internal string; you can then read it to get that string back. It looks like an IO object but acts like an in-memory string. One aspect that’s interesting is the original `after_teardown` bind self call. What we’re doing here is grabbing the instance method `after_teardown` off the Action Dispatch system testing helper.

00:27:38.980 The reason we do this is that Action Dispatch hook is set to execute when the test finishes. However, RSpec doesn’t print failure output at the same time. When you look at an RSpec failure trace, its output is at the end of the entire test run, not inline. Thus, we have to reorder how these outputs are printed. We accomplish this by saying take this method, rebind it to me, and invoke it. The way we get those failure lines into RSpec output is through a pre-existing protocol. Thus, I didn’t even need to enter the RSpec core and rebuild that part of the framework.

00:28:20.770 Finally, we need to ensure the Rails `after_teardown` hook is never called. We accomplish this by defining a method to blow away the `after_teardown`, which literally does nothing. Once we include that when we’re incorporated, it’s almost like monkey patching. We additionally need to communicate to Action Dispatch whether or not our test has passed. While it’s straightforward for a MiniTest test to know its status, it’s more complicated in RSpec.

00:29:11.370 The Rails team does a wonderful job of designing a robust inner layer of API beneath the framework that Rails users call. This makes my role as an external library developer easier because Rails provides great, rich APIs that can be integrated into RSpec without too much difficulty. As much as I’d like to claim that this integration was straightforward, a major feature release like this is bound to encounter bugs. One of the very first things I did after getting that pull request up was reach out via my network and social media, asking folks to give it a try. Since Derek had expressed interest in browser testing, he immediately reported every problem he encountered.

00:30:12.780 We had issues with using `feature` instead of `describe`. Please fix that! It turned out to be impossible: I couldn't set the driver in a global before hook, which is why I'm calling before in every single test. So a global hook couldn’t override that situation. Later, we added a screenshot function; initially, the screenshots just got saved as ‘screenshots/failures/’ and nothing more. The reasoning behind this was interesting.

00:30:48.099 MiniTest relies on the method name that's currently being invoked to identify your test. Thus, all tests are methods, and the way they retrieve this information is by using the method name. Anyone who is well-acquainted with Ruby won’t shy away from monkey patching; I just went ahead and overrode the method name on an RSpec test to provide surprisingly good human-readable descriptions of your tests. It takes the class name, typically like `RSpec.example_groups`, and a description of your test while replacing spaces with underscores and performing natural language scrubbing. We also add ‘R’ and a number like ‘1000’ at the end so if you run the same test multiple times, you may want screenshots from different runs, which helps make them unique.

00:31:55.430 This leads us to the final item: the ultimate no-no for RSpec. Puma printing stuff in your success output. I cannot have any output in my green dots; it's unacceptable! Fixing this required me to dive into Rails because that’s where the Puma configurations reside. While implementing this feature, I merged a pull request in Rails to facilitate this. This is one of those great Ruby open-source moments I’d like to highlight: I’m a third-party maintainer without authority over the Rails team. I motivated to create this pull request, wrote a simple patch, and while I’d like to say there was discussion, I sent it to Sean, and he merged it immediately.

00:32:37.379 Finally, the worst kind of event occurred when we released a new feature in a gem like RSpec: we broke people's test suites. This primarily impacted users on Rails 5.1 who were utilizing this feature. The reason stemmed from a load order issue where we were assuming that you would have Capybara included, but people didn’t. Consequently, their entire test suites would fail — which is a massive problem. We decided it was fine to rescue load errors and provide a more helpful error message. The point of this is clear: open-source software is significantly imperfect.

00:33:32.209 We are all people working on our spare time, unpaid, usually on evenings and weekends. RSpec maintainers typically don’t get paid to maintain it. The MIT license offers us a get-out clause for this: it states that software is provided as-is, without warranty, including the implied warranties of merchantability and fitness for a particular purpose. This means I can break everything you do, and you can’t sue me. That’s how it works, and that’s all I have.

00:34:00.000 Thank you so much! Just a quick shout-out to my cool team from DigitalOcean.

00:34:31.240 It’s a funny story: I was a developer on my own for about nine months and didn’t have a team because we were trying to recruit. Then, I went to RubyConf and immediately found two developers to join my team. I love hiring from this community because you are my people. Even while we don’t work with Rails as much at DigitalOcean, it remains a significant part of our stack. I think it’s amazing that after being in this community for so many years, I could hire people I genuinely enjoy working with.

00:35:01.990 Shout out to Chris and Kinsey if they are here, and a shout-out to Dan, who wasn’t able to make it but will probably watch this talk at some point. We have a booth — check it out for swag and a demo with a giant button, which is really cool! I will actually be down there after this, happy to take your questions and chat with you.

00:35:39.000 That’s all I have! Thank you so much! I’m Sam Phippen on Twitter; you can email me as well within DigitalOcean.

00:36:10.000 You

RailsConf 2018