Ruby Video | Teaching Capybara Testing - An Illustrated Adventure

Capybara

Teaching Capybara Testing - An Illustrated Adventure

Brandon Weaver

Teaching Capybara Testing - An Illustrated Adventure

by Brandon Weaver

In the video titled 'Teaching Capybara Testing - An Illustrated Adventure,' speaker Brandon Weaver discusses the significance of end-to-end testing with Capybara, presenting it through an engaging narrative with illustrative elements. The story follows Red, a lemur who believes he has achieved complete test coverage for his Ruby application. However, his world shifts when Currant, a magical Capybara, arrives to challenge his understanding by introducing the concept of end-to-end tests.

Key points covered in the video include:

- Types of Testing: Brandon explains three types of testing: unit, integration, and end-to-end tests. Unit tests focus on individual code units, while integration tests assess how these units work together. In contrast, end-to-end tests simulate a user's experience by testing the application as a complete system.

- User-Centric Approach: The importance of end-to-end testing is highlighted through an example where a user encounters errors while trying to use the application, underscoring that ultimate user satisfaction hinges on thorough testing of critical flows with Capybara.

- Getting Started with Capybara: Brandon provides a step-by-step guide to setting up Capybara tests, using Selenium for browser interaction, and emphasizes proper configuration, including running tests in incognito mode to avoid unintentional carry-over effects like cookies.

- Creating End-to-End Tests: The video walks through the process of writing an end-to-end test for user registration and the login process. This includes using various methods like 'clickon' and 'fillin' to interact with web forms and verifying outcomes through expectations.

- Addressing Brittle Tests: Brandon explains that tests can often become brittle due to structural changes in the application. He emphasizes writing tests in a human-centric way, favoring querying by role or label over more fragile methods like using CSS classes.

- Community Values: The presentation concludes with a reminder of the importance of community support in tech, encouraging inclusivity and reflection on the treatment of individuals within the community.

Overall, the video presents an informative, engaging lesson on the necessity of end-to-end testing for ensuring web applications work flawlessly from the user's perspective, balancing technical instruction with narrative storytelling.

00:00:19.560 I'm so glad you could make it! Grab a seat, and get comfortable, because we're about to get started.

00:00:25.859 So first, who am I? Well, simply put, my name is Brandon, and I work on a team at Gusto called Service Modularity. But we are here more to talk about Capybara, so let's go ahead and just get into that, shall we?

00:00:43.860 We begin with a curious case of a Capybara named Krant. You see, years ago, Red had an encounter that would change the way he thought about testing.

00:00:51.000 It was after one of his proudest moments working with his new application—a way to decide where in the world to go to dinner after these conference things when they wrap up for the day and who to go with.

00:01:01.620 He was proud because he had achieved 100% test coverage. He was elated, dancing around the room, and feeling invincible because he had proven that his application worked. Nothing could go wrong, right?

00:01:14.220 All of this joy came crashing down when he heard a startling sound at the window. Turning to see what it was, he was greeted by a strange sight: a Capybara in magician's robes.

00:01:27.360 "That's indeed quite the application, young one, but may I ask, where are your end-to-end tests?" Red stared, confused for a moment, trying to figure out what this Capybara was doing in his window.

00:01:38.820 He had to ask, "What's an end-to-end test?" and "How in the world did you even get in here? This is on the 12th story!" With a grin and a laugh, the strange Capybara answered, "Oh, nothing terribly surprising. I just snuck in through the gaps in your unit tests. You'd be surprised how much you can slip through even with 100% coverage."

00:01:50.939 As for your first question, I believe I may be of some help: you may call me Krant, and I’m here to teach you about end-to-end tests. Red was not quite convinced by this strange Capybara, but he found himself ever more curious as Krant began to share knowledge about different types of testing.

00:02:22.140 To start with, there are three types of testing we should concern ourselves with: unit tests, integration tests, and end-to-end tests. Unit tests, which you're probably already familiar with, focus on isolating and testing individual units of code—much like building blocks.

00:02:47.640 You'll have the most unit tests, and they are also the fastest to run since they focus on very small areas of code. The downside is that unit tests can be too focused and heavily stubbed, which means they may not accurately reflect the real behavior of your application.

00:03:04.379 Next, we have integration tests which focus on the integration of several units and how they work together—similar to how several building blocks can create a larger structure, say a wall. You might use them when testing things like controllers with request specs or model scopes interacting with the database.

00:03:27.900 However, that's not why we're here today; we are here to talk about end-to-end tests. If unit tests are the building blocks and integration tests are the basic structure, end-to-end tests represent the whole building, including the water, electricity, and the people running around inside.

00:03:50.819 They describe how your application works from the user's perspective, not just from your code. It’s curious how they may compose the smallest part of this pyramid, but despite being the fewest in number, they are likely the most valuable to your user.

00:04:09.599 To prove that your site works as expected, let’s consider an example: suppose a user wants to join a group on your site. They want to go to supper but encounter a 500 error, or worse—nothing at all! Just a blank page with no error or message, leaving them with no idea what went wrong.

00:04:34.440 If you respond to that poor lemur by saying you have 100% test coverage, I don't think they would care. What they want instead is their supper. At the end of the day, it doesn't matter whether the site is covered by unit tests; the users want the site to function as intended.

00:04:55.139 That's why we write Capybara specs—to cover those critical flows of the site and ensure things actually work. However, they are not exactly free.

00:05:07.500 In fact, you might notice that you write a lot more unit tests than end-to-end tests; that's to be expected since there are many more units of code than there are flows in the application. End-to-end tests can also be expensive and create a lot of data; sometimes they can be slow or flaky.

00:05:19.380 I contest that last point, but we can focus on that later. For now, let's get into some testing.

00:05:30.000 Part two: our first end-to-end test. To start, we need to ask ourselves: what should a user be able to do with our site? What do we expect them to be able to do?"

00:05:49.680 Well, I suppose I'd like to make an account and find a group to go to dinner with. I haven't done much design yet; it's just plain HTML. What if we started there?

00:06:01.740 That sounds like a lovely start! In fact, starting with plain HTML will keep us honest later and make our tests more robust as we add styling and classes.

00:06:14.580 But first, we need a bit of configuration so that I know how to interact with your site. We'll start by configuring Capybara to work with your code.

00:06:32.619 Often, this is done through Selenium, which allows me to interact with web pages. You'll probably want to see what I'm doing while testing, so you’d want a configuration that runs in a browser—preferably in debug mode.

00:06:51.930 The first thing we do is register a new driver which we can call 'selenium_debug_chrome', and by the end, we'll set this as our default driver. We want a few options for our new driver: running it in incognito mode so we don’t have anything weird carrying over—like cookies—and disable extensions to prevent odd side effects.

00:07:14.580 Next, we want to ensure that our developer tools are open, and the window size is something reasonable. We can change the size later, but this gives us a base to work with.

00:07:26.760 Next up, we want a way to turn on headless mode in case we don't want the browser running; we can do this with an environment variable named 'headless'. Most of the options won’t apply here. Why disable GPU? Windows has some quirks that we have to account for.

00:07:43.560 At the end, we make our new Selenium driver using the Chrome browser with our options. Don’t worry if you didn’t catch all that; it’ll be in the notes whenever we're done here.

00:07:58.680 We need a few things to get going. First, we want this to run in head mode to actually see what's happening.

00:08:12.120 Whenever we start, we set up a basic structure that’s going to look very similar to your RSpec test. Capybara primarily runs in RSpec unless you're doing something different.

00:08:30.000 We’ll create a folder and call it 'specs/system'. You could also name it 'spec/features' or 'spec/end_to_end'—it doesn't matter. It's up to you. But let's start with the registration and login test just to prove that we can actually create an account.

00:08:48.780 Notice I have a single method call here: 'visit'. Why? Well, Capybaras are quite lazy, and I don't want to guess. I want to know exactly how to do this.

00:09:00.720 So how exactly do we do this? How do we know what to write for this web page? We stick a 'binding.irb' in there and step through it piece by piece right below the visit. This puts us in the context of our test.

00:09:13.200 It means we can interact with the web page from our IRB console and figure out exactly what this should be doing. Let's skip to a new view where we have the browser on top and the console down below. We’ll keep dev tools closed for the moment, but they may come in handy later.

00:09:32.160 Let's start by clicking the sign-up link. If you notice, it's up there in the top left, and we can use something called 'click_on' to look for something clickable, whether it be a button or a link.

00:09:48.480 Interestingly, Capybara is very patient. It will wait at least two seconds by default unless instructed to wait longer to ensure that the link is present on the page before proceeding.

00:10:03.360 Now let’s press enter and see what happens. It brings us to our registration form. Now, seeing this form, what exactly might you want to do here?

00:10:19.020 I suppose I want to fill in the name, email, password, and confirm password fields. Well, let's do just that.

00:10:36.300 As a student, we have four fields that should be filled in: name, email, password, and confirm password. We can use 'fill_in' to provide content like 'name here' and press enter.

00:10:50.460 What's interesting is that form labels are sufficient for Capybara to identify which fields should be filled in. You’ll notice that the text is now reflected in that field.

00:11:06.240 Now, let's do email next and press enter. Oh, that’s curious! Right, it used to be called 'login', but I felt that wasn't very clear, so I renamed it to 'email'.

00:11:22.640 Let’s check the web console real quick to confirm our changes. Notice this little arrow box in the top left corner of your Chrome window within developer tools.

00:11:38.520 If you click on that, we can navigate to the email form field and check its structure. We'll find that this field is still referencing 'login' in multiple attributes.

00:11:52.560 So, let's update its corresponding values quickly to reflect this change. Now let's run 'fill_in' again, and while it works, we must remember to resolve the underlying code.

00:12:06.540 So, why do we care if the email matches? Since a screen reader user would be very confused trying to decipher this, it’s critical.

00:12:20.880 This leads us to a vital point: accessible code is testable code. Notice how we've used names and roles to implement useful actions like 'click_on' and 'fill_in'.

00:12:38.520 That's not a coincidence—the more accessible your site, the easier it is to test and the less brittle it becomes when styling is added.

00:12:57.900 Now getting back to it, our current test might look something like this. We're going with the behavior we've already implemented and keeping the 'binding.irb'.

00:13:13.440 This particular approach is not test-driven development; it's what we call rebel-driven development. This technique can be very handy when dealing with existing code to test and evaluate functionality.

00:13:34.920 Alright, we still have the password and confirm password fields to fill. I suggest our security-conscious friends look away—these are not production passwords, I promise!

00:13:51.480 Once we input those fields, we can click the create account button. Now, after success, you can observe two potentially interesting items on the page.

00:14:04.920 Near the top, there should be a flash message stating, 'An email has been sent to verify your account,' and you might also notice the name of the new account in place of where 'sign in' used to be.

00:14:19.920 These two are what we can write expectations against to confirm that this workflow was successful. Just like anything else in RSpec, we can test them.

00:14:37.440 We can expect the page to have content saying 'An email has been sent' and ensure it includes the correct username.

00:14:54.480 With that, we have our spec written! We can now run it after filling in the rest of the test.

00:15:08.760 Back to the spec, we'll see that it runs successfully. However, you may wonder, why are we testing this? Didn't Jeremy Evans do a good enough job testing Rails?

00:15:20.640 Of course he did! He’s quite skilled, but he cannot guarantee how that particular feature is functioning within our application context.

00:15:37.800 We want to test this flow to ensure it still works effectively, giving us substantial value in the process.

00:15:50.760 Let’s take a look at another flow and enjoy a convenient break.

00:16:00.600 In general, you should have a test for every major flow of your application. Logging in, signing in, and registering are definitely among those flows.

00:16:14.940 If only to ensure they function as expected and you haven’t broken any critical flows through recent changes.

00:16:26.820 Whether or not you technically own that code, it’s part of your application and should be tested.

00:16:40.380 You should especially test those cases. Why aren't those covered by the authors of the gems? Yes and no—trust, but verify!

00:17:02.640 You cannot guarantee that this particular library will function in the context of all the gems you're using. They might not know subtle ways a loop could break if you happen to include a specific gem.

00:17:11.760 At a high level, we want to at least ensure that this is working, which underscores the benefits of having a Capybara like me around.

00:17:25.800 We should always focus on our own code. These high-level flows are what users expect from our application, and we should ensure they work as intended.

00:17:38.400 Let’s consider a high-level flow for a user account joining a group headed out for supper after a conference.

00:17:48.300 That was a lot of data modeling and terminology, so let’s take a step back. What does our data model look like?

00:18:01.380 We start with a gathering, like let's say RailsConf, which has many outings. The outings might include something like supper tonight and it has many accounts which could represent the participants of that conference.

00:18:16.740 These outings could be supper tonight, karaoke, or any number of possibilities. We don’t know how many attendees to expect.

00:18:30.900 To help with this, we can create a login helper to assist with the authentication processes, as we will be testing things repeatedly.

00:18:42.840 Thus, we can create a user, log in, and use a factory that can emulate a gathering—possibly a gathering called RailsConf 2023!

00:18:56.460 Now, given these aspects, let's fill out some of the setup code quickly and leave a 'binding.rb' for our next run.

00:19:07.200 We need an account to log in with. Also, let’s ensure that our supper gathering is in Eastern time because time zones can be tricky!

00:19:21.300 We certainly don't want to be testing in Pacific time for an event occurring in Eastern time—take it from me!

00:19:32.040 Now let’s create the outing, which would be RailsConf 2023. Next, we want to visit the homepage, log in, and focus on what exactly we are trying to test.

00:19:46.200 In this case, we want to confirm that a user can join a group for an outing. We'll start from the front page after logging in.

00:20:01.320 We want to click on the gathering name 'RailsConf 2023', which is a link, so we can use click_on to navigate to it.

00:20:17.700 Now we should probably pretty this code up later, but lets just hit the join group button and see what happens.

00:20:33.300 Okay, it looks like the user indeed can join a group. But what’s with the names here? Explain it to me, I’m confused!

00:20:48.840 Oh, right! I thought it would be fun to generate a unique name by combining two adjectives, a color, an animal name, and part of a uuid to ensure it's distinct.

00:21:05.520 Well, I understand your intention with the name, but it could be clearer what that function serves on the page. Perhaps you could prefix it with something like 'Group Name'!

00:21:18.480 That way, screen readers won’t be left wondering why there is a 'Hilarious Emerald Jackal' appearing on the page.

00:21:32.520 Moving on, let’s write a few expectations to ensure that we have a visible list of members and that our account shows up on the page.

00:21:44.640 An interesting issue arises here: the email address appears in the header if the user doesn’t have a profile name, as well as in the members list.

00:21:58.680 How do we guarantee that the email we’re seeing is actually in the members section and not in the header? You can’t just ask me to search the entire page! I can't assure you that I’m looking in the right area.

00:22:11.640 Is there a method to tell Capybara to specifically look? Yes! With semantic HTML, we can see that there’s a header and a main section.

00:22:24.480 The main section is what we're primarily testing against. Let's use 'within' to ensure we’re scoped correctly and only searching in that area.

00:22:40.560 But be careful not to abuse this method and avoid using CSS classes when possible, as they can become brittle, and Capybara can become temperamental.

00:22:55.980 Instead, try to query as a person might use the page, not as a machine. Now, we want to ensure to retain a hold of that header text from the previous page.

00:23:12.360 We should remind ourselves that if you add a prefix of group name, the text will be different, so you'll need to clean up that text to avoid breaking the specs.

00:23:25.800 Let's return to the outing page because if you just joined an outing, then you should probably not be able to join a new group!

00:23:38.400 You should know which group you are in, so let's click back to the outing to see if our expected behavior is functioning as intended.

00:23:54.180 Again, it wouldn’t make sense to join a group you're already a part of, and we want to make sure that this behavior works properly.

00:24:08.760 We can set a prefix like, 'Your group is' and show exactly where the user is. Eventually, we may want to keep this admin-only, ensuring this test isn't accessible to unauthorized users.

00:24:25.560 The last two expectations here might be that the group name is visible, and that the join button is absent because we want to prevent users from joining multiple groups.

00:24:41.880 Altogether, this might look something like this. Personally, I prefer to describe individual sections of the page I'm testing, but modals may complicate that, especially if they don't exist in the main elements.

00:24:54.720 This is especially valuable in cases of JavaScript or CSS bundlers that may insert content into the page head, so I like to ensure I read all of my code.

00:25:08.520 I also prefer to normalize whitespace for readability. If you’re asking me to read a zoo of 12,000 lines of JavaScript and CSS, you'll find I will be staggeringly slow.

00:25:23.760 This is all well and good, but one complaint I hear frequently is about brittleness. Let’s explore that and see how we can prettify the site while we’re at it.

00:25:39.660 Part four: on brittleness.

00:25:54.240 Now, remember earlier when we discussed how well-written Capybara tests avoid breaking on page structure changes? Let’s suppose we revamped the entire site using a framework like Bootstrap, Tailwind, or Bulma, which substantially alters the page structure, introducing a significant amount of HTML boilerplate.

00:26:05.520 Our tests should still pass despite these changes! Now, let’s say we give it a quick styling update that makes the site look completely different.

00:26:18.599 Items are rearranged, placed in different positions on the page, and the number of HTML elements has increased since we now utilize more CSS classes.

00:26:33.300 What do you think will happen when we run the tests after this complete restructuring and reordering of content?

00:26:47.220 Will Capybara still be brittle? I would expect these tests to fail, but let’s run them and see.

00:27:04.920 Not a single failure—interesting, isn’t it? Why do you suppose that is?

00:27:15.240 There’s something fascinating at play here! Maybe Capybara isn’t as brittle as I once thought.

00:27:26.760 To be fair, this can depend on how you craft the tests. There are principles that help significantly reduce brittleness, particularly ideas that arise from front-end development.

00:27:39.859 I don't claim to be an expert on this, but I’m learning. I’ll give you a brief overview of how these ideas prioritize finding information on a page.

00:27:54.579 As we are a bit short on time, the central idea revolves around interacting with a website like a human would—or in this case, like a lemur or a capybara.

00:28:10.560 The core concept is about locating information on a page. Take our account creation page after its styling; as a person, you’d focus on filling in the name, email, password, and confirm password fields before clicking 'Create Account'.

00:28:28.320 If you think about it, this is how we would approach it in Capybara. How does this work? It’s because actions like 'fill_in' and 'click_on' depend on key types of queries.

00:28:43.080 They involve finding elements by their label or by their role—either of which are easily accessible queries.

00:28:58.920 There are more types, and I’ll provide references to the React Testing Library principles soon.

00:29:06.960 To summarize: the first group of accessible queries, in order of preference, includes finding by role, label, placeholder, text, and display value.

00:29:25.440 Each of these has a corresponding Capybara equivalent, and we have a testing site to experiment with this.

00:29:40.320 These initial methods are considered the best way to capture tests as they provide the highest accessibility to users.

00:29:55.720 But the remaining semantic queries may vary among browsers and screen readers, often relying on ARIA properties or HTML5 semantic tags.

00:30:06.840 You might also search by attributes like 'alt text' for images or 'title' roles for elements on the page.

00:30:22.920 Lastly, there are test IDs which help target tests via CSS, specifically for testing purposes rather than styling.

00:30:35.640 This approach is suggested only when all other options have failed. I encourage use of accessible queries first to prevent brittleness.

00:30:47.640 Even as a newbie Capybara user, I recognize there's a small subset of capabilities that need exploration—there’s so much more to learn!

00:31:00.780 Hopefully, this provides a solid starting point for your own journeys with Capybara.

00:31:13.020 So Grant the Capybara finished his lesson, leaving young Red with much to think about and an exciting journey ahead.

00:31:24.480 Time flows on, and so Red the lemur and Grant the Capybara continue to learn and grow together, as we all do.

00:31:34.980 No matter where we are on our paths, we always have something to share and learn, with a journey ahead.

00:31:46.920 The wonderful Ruby community will always welcome us on that journey.

00:31:57.840 As we wrap up, between the illustrations, the application behind this talk, and everything else, it’s incredible to be on the other side of this experience.

00:32:11.040 If you’d like to know who I am or find any links or resources, you can reach me on various social networks such as Mastodon, GitHub, Dev.to, Discord, or Slack.

00:32:26.040 Feel free to connect with me in the RailsConf Slack if you’d like to say hi!

00:32:41.040 Now, please note there’s a big disclaimer here: this application, which would allow finding supper tonight, isn’t production-ready.

00:32:55.680 It may sound cool if it were fully operational, but I can't create both a production-ready application and an illustrated talk without losing my sanity!

00:33:12.000 Maybe by RubyCon, who knows? If you're familiar with graph theory, particularly regarding social networks, let's chat!

00:33:27.120 If you want to see what I’ve been up to, it's open-source; certainly a lot of work remains.

00:33:41.040 There is a lot of green pathing to navigate because this was all generated from a conference talk.

00:33:54.540 Before we conclude, I’d like to thank everyone involved—nothing would exist without the wonderful community members who encouraged, supported, and reviewed my work.

00:34:42.960 Community is delicate; we must protect those who might be harmed, intentionally or unintentionally.

00:34:59.520 How we treat the most vulnerable in our community is a reflection of who we are and what we value. We must reject all forms of hatred.

00:35:07.440 With that said, that’s all I have for today. Thank you!

RailsConf 2023