Automated Testing

Summarized using AI

Accessible by default

Joel Hawksley • April 24, 2023 • Atlanta, GA

The talk titled "Accessible by Default" presented by Joel Hawksley at RailsConf 2023 focuses on overcoming accessibility challenges in software development, particularly in longstanding applications like GitHub's Ruby on Rails monolith. Hawksley shares personal experiences related to accessibility, emphasizing the importance of designing for users with varying disabilities. He discusses three main strategies GitHub employs to enhance accessibility: automated accessibility scanning, a custom form builder, and a process called preview-driven development.

  • Personal Anecdote on Accessibility: Hawksley relates a personal story about a natural disaster that affected his family, leading to a newfound understanding of situational and cognitive disabilities.
  • Defining Accessibility: Accessibility is defined as the ability for a product or service to be usable by everyone. He highlights that disabilities can be permanent, temporary, or situational, with many aspects of accessibility affecting a wide range of users.
  • Automated Accessibility Scanning: To confront accessibility issues proactively, GitHub leverages automated accessibility tools such as axe for continuous integration, integrating accessibility checks into their development process.
  • Custom Forms Framework: Recognizing that forms are fundamental in Rails, a new accessible form builder was developed to streamline form creation while ensuring compliance with accessibility standards, thus minimizing common errors in form design.
  • Preview-Driven Development: This innovative approach allows developers to create component previews, which streamline testing and facilitate immediate accessibility feedback during development.
  • Universal Design Concept: Hawksley touches upon the idea that making software accessible benefits all users, not just those with disabilities, using curb cuts as an analogy.
  • Empathy in Engineering: He emphasizes the necessity of empathy in engineering practices, urging developers to recognize and address the diverse needs of users.

The major conclusion of the talk is that accessibility in technology is not only a necessity but a fundamental human right, and innovative practices must be integrated into the culture of development to ensure inclusivity. GitHub's commitment to enhancing accessibility reflects its core value of participation and contribution from all users, regardless of their abilities.

Accessible by default
Joel Hawksley • April 24, 2023 • Atlanta, GA

It's one thing to build a new application that meets the latest accessibility standards, but it's another thing to update an existing application to meet them. In this talk, we'll share how we're using automated accessibility scanning, preview-driven development, and an accessibility-first form builder to make GitHub's 15-year-old, 1,400-controller Ruby on Rails monolith accessible.

RailsConf 2023

00:00:24.000 It's great to see you all! It’s so nice to be at an in-person RailsConf. COVID was hard.
00:00:29.699 Before I get started, I want to give just a brief warning. I'm going to talk about some difficult stuff today, specifically about a natural disaster that affected my family.
00:00:35.460 It'll be gentle, but I figured I'd mention it.
00:00:47.460 In the summer of 2020, my wife, our dog, and I moved into a home where we thought we were going to raise our family. Like many in the industry, I would say that I lived a really comfortable life. Some might even call it a life of privilege. We had very little to worry about; that all changed about a year and a half after this photo was taken.
00:01:00.960 We were on a road trip to San Diego, playing fetch with our dog as the sun set.
00:01:06.240 We got into the car to drive home and received a text message to evacuate our house back in Colorado. This was right around three o'clock in the afternoon. A couple of minutes later, I got some notifications from our security system at home that there was ash falling in front of our house.
00:01:17.880 I checked my phone; I had an air quality monitor in my house, and things were not looking good. This particular measurement, PM 2.5, is a general common measurement. This number was worse than the worst measurement ever recorded in Beijing, and this was in my home office.
00:01:29.700 About an hour later, I saw this picture on Twitter. This is a hundred yards from my house.
00:01:39.360 But we never saw any pictures from inside our neighborhood, at least during the fire. The following morning, I pulled up Twitter and found a video showing what our neighborhood looked like. Over a hundred homes burned to the ground; ours is kind of in the top right corner.
00:01:47.959 In our area alone, there were a thousand homes that were lost that day.
00:02:04.100 It took a few days before my wife and I could see it for ourselves. This is us in front of our foundation at that point.
00:02:15.000 One memory that really sticks out to me is realizing, two weeks after this, that I hadn't walked my dog since the fire—literally something I had done three or four days beforehand. I hadn't even thought of doing it for two weeks.
00:02:28.440 I noticed other things too. It was really hard for me to focus on anything. I got really angry, really easily. Something else that was really difficult was that the technology I used on a regular basis before no longer worked for me.
00:02:39.360 I remember particularly having a really hard time solving image captions. I got really angry when I had to click on which picture was a bus.
00:02:47.760 I had similar struggles with one password, too. I remember it being really hard just to remember how to use one password.
00:02:53.640 What I soon learned from someone at the Red Cross, who was helping a lot of us fire victims, was that this kind of significant trauma can actually cause brain damage. In my case, that damage was both temporary and, as I'm finding out now, permanent.
00:03:10.800 This experience really opened my eyes to the privilege I had before. It changed the way I viewed the world more than anything else in my life up to this point.
00:03:21.360 So that's what I'm going to talk about today, specifically about accessibility. Accessibility has many definitions, but the one I like is from the Interaction Design Foundation, which states that accessibility is the concept of whether a product or service can be used by everyone, regardless of how they encounter it.
00:03:39.240 In the past, I imagined that people with disabilities were mostly those using wheelchairs or blind people using canes. However, I have since learned that disabilities come in many different forms. One framework we used to describe this is situational, temporary, and permanent.
00:03:53.759 For example, a permanent touch disability might be having one arm; a temporary touch disability could be having a broken arm; and a situational touch disability could be holding a baby or even just holding a door open for someone while you walk into a building.
00:04:05.279 For hearing, a permanent disability is deafness; a temporary might be an ear infection; and situational could be working in a loud environment, like if you're a bartender.
00:04:21.180 So when I couldn't complete these captions, what I had was a temporary cognitive disability. I'm now able to complete them again, but it took a while—about a month or two—before I was able to reliably complete them.
00:04:30.420 There are various guidelines and standards for accessibility, such as the WIC and Section 508 APG.
00:04:38.039 Just know that they don't even all agree with each other. However, in practice, for the work we do in accessibility, it means that our applications work with assistive technology.
00:04:51.060 We mainly focus on screen readers, which are tools that read the screen out loud via keyboard control. For example, when a screen reader looks at this GitHub profile page, VoiceOver announces each section of the page as the user navigates using keyboard navigation.
00:05:06.339 Effectively, screen readers turn web pages into a tree structure that is navigated with the keyboard.
00:05:12.000 What's great is that as of a recent Chrome release, you can actually view this tree structure navigated by screen readers in the Chrome DevTools under Accessibility. If there's one takeaway from today's talk, it's that this tool is amazing and will greatly assist you in determining how accessible your application is.
00:05:31.320 Today, I'm going to share how we're taking this approach to improve the GitHub experience for users of assistive technology, like screen readers. We achieve this using automated scanning, a custom form builder, and what we're calling preview-driven development.
00:05:45.060 Hi, my name is Joel, and I’m a staff engineer at GitHub. I work on various company-wide UI-related initiatives, like accessibility. I'm kind of like a UI architect of sorts for the company.
00:06:03.600 I want to share details about what it's like to work at GitHub. I like to start by giving a rough idea of the scale of the company because I know that we're all at companies of very different sizes, and this helps you understand how what I'm talking about may apply to your situation.
00:06:20.160 As of December, we have about 4,000 employees. GitHub is still a monolithic Ruby on Rails application. I say that every year, and I think I'm going to be saying that for a while. When it comes to serving our customer-facing traffic, we have about 1,400 controllers in our code base. It's actually 1,435, so this has increased since I made this slide.
00:06:34.560 That's 25 more controllers than we had when I gave a talk at RailsConf last year, which is wild. I love mentioning this stat every year because it really shows how much we continue to invest in our Ruby on Rails monolith, even after so many years. To give you an idea, that 25 was 350 controllers in just one year.
00:06:58.620 I've worked on a ton of Rails apps that never made it to 350 controllers. This is just one year of GitHub getting bigger, and it's a 15-year-old code base, too. We're adding 350 controllers 15 years into building this app, which is mind-blowing.
00:07:15.240 Of course, to do that, we need millions of lines of Ruby. This is another example showing our growth: this is the number of lines of ERB in our code base over time, starting from commit number one sometime in late 2007.
00:07:31.560 It’s grown to around 400,000 lines now. You can see that this curve is linear and going up. I think it's important to share this context because it can frame why we make some of the decisions I'm going to discuss today.
00:07:43.560 Another important aspect I haven't discussed before is some cultural context of what GitHub is like as a company and how that might affect the things we do from an engineering standpoint. I think it's essential to share a little of our history because GitHub is kind of a unique company.
00:07:58.080 For the first six years that GitHub existed, we had no managers. Does anyone here work at a manager-less company? One person? Okay, I think that fad is pretty much dead at this point, but it was a big trend at the time.
00:08:16.320 When you went to our offices in San Francisco, the entrance was a full-scale replica of the Oval Office from the White House. This is no longer the case but was true at the time. On the floor there was a rug that read, 'The United Meritocracy of GitHub.'
00:08:30.600 For years, we operated in what some people called a 'cooking for chefs' mindset. You know, we were developers; we knew what we wanted GitHub to be, so we didn't need to ask anyone what it should be.
00:08:49.920 This meant we could build whatever sounded good to us and reasonably expect our customers would agree. I don't think the no-manager approach would have worked for as many years as it did if that weren't the case.
00:09:05.340 It's worth noting that we went about six years with no managers and nine years with no product managers, which was equally wild. However, as you might imagine, having no managers has its downsides.
00:09:20.220 I know of cases where we basically ignored requirements from customers because we couldn't convince anyone in the company to work on them, which unfortunately cost us sales. Thankfully, a lot has changed since then.
00:09:32.520 Today, one of our core company tenets is that we want to be the home for all developers, no matter who they are. When it comes to accessibility, our goal is pretty simple: full and equal participation for people with disabilities in the development process.
00:09:51.240 This is what we recently published as a company, saying we believe people with disabilities should benefit from and contribute to the creation of human progress. You all know, better than most people, that software is driving the future of the world, and we believe people with disabilities should be able to be part of that progress.
00:10:07.920 We also believe that access to technology is a fundamental human right that everyone deserves. These are big goals, especially when considering a surface area of thousands of unique user-facing pages. I think we have around 2,000 unique screens in our app.
00:10:23.340 Because of the scale and how quickly our products are growing, manually checking and fixing each of these pages for accessibility issues is basically impossible for us. Even if we went that route, you can imagine we'd fix a page, and then six months later, it might regress.
00:10:37.320 So, to achieve our goals, we really need some good tools, and one of those is automated accessibility scanning. If you take nothing else away from this talk besides what I've said earlier, know that bad accessibility almost never breaks CI. Very rarely do our standard practices in the industry lead to accessible experiences.
00:10:57.740 Even when we have automated accessibility checks, they're far from perfect. Depending on whom you ask, between 30 to 50 percent of the classes of accessibility issues can be caught with automation.
00:11:10.200 In the most optimistic sense, you might say 50 percent, but I'm in the belief that it’s actually lower than 30 based on my experience. For example, while we can check for alt text on images, we can't check that the alternative text correctly describes the image.
00:11:30.600 For instance, if the alt text says, 'A cat sitting on a rock,' but the picture is of a dog, then while the alt text exists, it incorrectly interprets the image for someone who is visually impaired.
00:11:50.000 There are also subtle issues; designers use techniques referred to as Gestalt techniques, such as proximity, where the grouping of items can imply meaning. This visual relationship is not conveyed to someone who is visually impaired by default.
00:12:05.760 That being said, automated scanning can be quite useful. The most common tool used in the industry is called Axe, which can run a couple of different ways. It can be run as a browser plugin, allowing you to view violations in your DevTools.
00:12:15.900 The most effective way we've found is to embed it in the page so that whenever a staff member runs our application, they can turn it on. It highlights accessibility issues in red on the page and logs them as console errors.
00:12:32.160 We also enable this as a bundle that staff members can use when they're working in production. Additionally, we use it in our end-to-end browser test suite that runs against every production deploy, built with Playwright. Our accessibility check hook runs Axe every time a page changes and fails the build if there are any Axe errors.
00:12:50.760 We also write custom Axe rules. This is particularly helpful because we have some patterns in our application that we know are inaccessible, but Axe by default won’t catch them. For example, if you hover over a button on GitHub and a tooltip appears, it's essential to ensure it's accessible.
00:13:09.960 Thus, the linter literally fails if the tooltip contains anything besides text. A big problem we’ve run into while rolling out Axe across our application is intractability.
00:13:27.420 In some cases, we’ve encountered UI patterns that were simply not accessible. Sometimes we had to rethink entire workflows, not just individual pages but how multiple pages connect together. UI interactions like drag and drop can be especially challenging.
00:13:45.000 However, despite these challenges, Axe serves as a great starting point for making your app accessible. Having all of your screens pass Axe is quite significant, likely placing you in the 90th percentile of accessibility.
00:14:02.760 So we generally start by making a page Axe clean before tackling the more labor-intensive work, as this helps reveal where some of the more fundamental intractable problems are.
00:14:16.740 Another critical tool in our accessibility work has been creating a custom forms framework. As many of you probably know, forms are the core competency of Rails.
00:14:32.800 If you go back to the original 'Build a Blog' YouTube video from DHH, there's a focus on building forms. 15 years later, he made another demo for Hotwire and it's still primarily about building forms.
00:14:43.320 Rails should excel at this, and it does. Rails has excellent built-in form helpers, such as form tag and form four. Looking at the example from the form helpers guide, you'll see how form with is essentially a one-to-one mapping with what comes out in the DOM.
00:14:56.100 It does add some important features, including CSRF tags, but at its core, it’s a DSL for DOM nodes. While this simplifies a ton of complexity around building forms correctly, it is not accessible right out of the box.
00:15:09.600 For instance, in this example code, the form input has an associated label generated by two separate method calls. This example does pass Axe's rule 4.4, but it can be cumbersome to remember to do this.
00:15:31.260 One of my favorite software engineering books is 'A Philosophy of Software Design', which discusses the importance of making invalid states impossible.
00:15:46.920 About a year ago, we realized we needed to apply this principle to how we build forms at GitHub. With many pages having forms, we hired an engineer eager to solve this problem.
00:16:02.640 I challenged him to find out what it would take to make the forms we build at GitHub accessible by default, hence the title of my talk. That's where our custom forms framework was born.
00:16:19.440 We architected a form builder to avoid many common accessibility bugs. In this new DSL, there’s now a single method to create a text field with a label instead of two separate methods. Under the hood, we generate both the input and label tags and correctly associate them according to Axe rule 4.4.
00:16:36.600 What we've effectively done is make it nearly impossible to violate Axe rule 4.4 when using this tool. While we can’t automate our way out of every possible accessibility violation, we ensure that this rule won’t be an issue for our customers anymore.
00:16:54.000 This work has taught us that focusing on accessibility helps us decide what components we should build because it reveals which abstractions enable accessibility.
00:17:08.760 For example, in our design system, we have an avatar component, which is prevalent throughout our application, used in various places.
00:17:24.360 The issue arises when this avatar component appears next to a text link to the user's profile. Sometimes, both elements use hover cards, which can cause screen readers to read the user’s name multiple times.
00:17:36.240 To resolve this common accessibility issue, we created a UI component called a nameplate. This contains both the avatar and the text link while working perfectly for screen reader users.
00:17:57.960 The last tool I want to discuss is called preview-driven development. How many people here are familiar with Action Mailer previews? About half? Okay.
00:18:15.360 Action Mailer previews are local development tools that allow you to develop and test your HTML emails in a local browser, which is nice and isolated.
00:18:29.220 We realized we should apply the same principle when we build our UI components, allowing us to write previews for them as well. Each preview is just an example of a component rendered in a specific state.
00:18:44.760 For instance, here's a preview for rendering a danger button. Previews for UI components look much like mailer previews, which are also controller-like.
00:19:00.840 For a button component, we may have a preview for each color scheme option and different functional modes defined by adding methods to that preview class.
00:19:15.600 We use these previews in our tests. We created a render preview helper that renders the preview code inside our component unit tests. This bypasses the browser and creates Capybara nodes for the rendered result, allowing us to assert against them.
00:19:31.560 This makes our UI tests easier to debug since you can often just look at the preview and see what went wrong without needing to boot up the app in a complex state.
00:19:47.880 So, if, for example, I misspell a selector in the test, it will fail. Instead of digging through rendered HTML, I can view the preview in Chrome and inspect its state.
00:20:05.520 Additionally, these previews make many of our UI components discoverable. We have thousands of components, and this organization helps prevent teams from inadvertently building the same thing.
00:20:23.520 We use a tool called Lookbook, similar to Storybook, to organize our previews into a kind of component directory that can be browsed in local development.
00:20:41.760 Lookbook works for Rails partials, too, so it can be used even if you aren't utilizing view components. It’s built from the ground up for Rails using Hotwire.
00:21:00.180 This conceptually couples the tests we write to the documentation. Developers can access this documentation, giving us an incentive to write practical test cases that serve as examples for others.
00:21:11.520 One side effect of this is that we used to have a separate documentation website for all our components and UI elements, which was somewhat manually created. We have since eliminated it and now use Lookbook instead.
00:21:23.700 The most crucial aspect of this workflow is that it leads developers towards running their components through Axe, as we've added accessibility scans to our Lookbook configuration.
00:21:38.740 When we build components in isolation, we now receive live accessibility feedback in the accessibility tab we've added in Lookbook.
00:21:51.120 We have a job that runs through all of the Lookbook previews, scanning them with Axe in CI. This workflow was one of the reasons many at GitHub wanted to adopt React wholesale.
00:22:03.180 While we're still migrating to react in bits and pieces, this shows how the Rails ecosystem can benefit from adapting ideas from other languages and frameworks without needing to rewrite our entire front end.
00:22:15.640 In accessibility practice, there’s a term called Universal Design. Universal Design posits that accessibility accommodations benefit everyone.
00:22:30.960 For example, curb cuts, originally intended for wheelchair users, also benefit people pushing strollers. These design choices create a more inclusive environment for everyone.
00:22:42.960 What might Universal Design look like in software? One great example is single sign-on, which allows people with disabilities to avoid captchas, known for being notoriously inaccessible.
00:22:55.440 After my own experiences, I began using sign in with Apple or sign in with Google to avoid captchas. This realization inspired me to change my habits.
00:23:09.720 The larger lesson here is that the examples and tools I've shared today are just part of a larger class of blind spots that many typically able, wealthy, first-world software developers have.
00:23:22.500 This reminds me of a quote from Donald Rumsfeld: we know there are known unknowns—things we know we don’t know—but there are also unknown unknowns—the ones we don’t know we don’t know.
00:23:35.040 History shows that it's the latter category that tends to be the most challenging.
00:23:49.680 Applying this thinking has opened my mind to similar problems. For example, at GitHub and many companies, we are issued powerful laptops.
00:24:03.840 These computers provide us with significant productivity boosts, resulting in a false sense of reality. Our experiences using our products differ from those of our customers.
00:24:17.280 For example, this benchmark from late 2021 shows that GitHub users were issued iPhone 13 Pros, while the next fastest Android phone was three times slower in JS performance.
00:24:30.600 A similar issue exists with request latency; most traffic is served from Virginia, which means our majority U.S. staff does not experience the same latency that users from around the world face.
00:24:43.920 So, how do we look out for these blind spots? We can do this by embedding empathy into our engineering practices, starting by putting ourselves in the shoes of others, especially those different from our own.
00:25:09.240 That's our neighborhood now; it's all coming together.
00:25:20.640 I have some time for questions. The first question is whether anything I've shown is open source. I should have included a link.
00:25:31.740 The forms work is all in the Primer View Components Ruby gem, so you can see it there. The Axe custom rules I mentioned? As of a couple of weeks ago, Axe GitHub is the owner and repository.
00:25:47.520 As for how quickly we get feedback in CI for the accessibility tests, it depends on which tests they are. Some can run statically, and those can be really fast.
00:26:05.640 But when it comes to browser-based testing, it can be slow. Playwright has proven really useful, and I find writing these tests much better than the Capybara stack.
00:26:14.520 I think Playwright tests run significantly faster, are less flaky, and make better use of machine resources.
00:26:23.520 Regarding colorblind-friendly options, it’s helpful to hear you ask that question. Our team shipped that a few months ago, but we might not have publicized it enough.
00:26:36.840 If you go into settings, you can find different color themes, including dark mode or configurable high-contrast themes.
00:26:52.680 The next question is about using AI tools to check captions.
00:27:06.240 Someone on our accessibility team has explored that; however, we don’t actually have a lot of images. This paradox exists since GitHub contains much user-generated content.
00:27:22.260 We have considered providing more feedback; perhaps we can prompt users to put alt text on images. AI could provide feedback, but it’s tricky.
00:27:37.560 The next question is about what caused GitHub to invest in accessibility. Unfortunately, there are limitations on what I can share regarding that.
00:27:53.640 One driving factor has been legal pressure, which is often a recurring theme in accessibility talks. If you haven't faced it yet, you will.
00:28:08.760 I encourage you to adopt Axe in your applications; automated tools can expose many accessibility issues, making you a target for complaints.
00:28:21.000 Next, we need to consider how to approach accessibility standards for user-generated content.
00:28:35.640 There are guidelines acknowledging that, as an application owner, it's not always within your control to enforce standard practices.
00:28:48.960 That said, if you use specific authoring tools, you can prompt users to create accessible content.
00:29:02.400 Our goal at GitHub is to ensure the app works for users with screen readers across all 2,000 pages, but that’s only the first step. Eventually, our goal is to lead in the space, ensuring accessible experiences.
00:29:15.840 Lastly, the question pertains to why we're migrating to React.
00:29:30.120 One interesting factor is hiring—finding enough people to build UI as we continue to add controllers becomes challenging.
00:29:40.920 Additionally, as we face complex UI challenges, React often proves to be beneficial for development.
00:29:55.200 Despite this shift, 97% of GitHub is still in the Rails server-rendered stack; I can’t foresee dropping below 90% in the near future.
00:30:07.640 The final question is about accessibility for APIs. While I haven't addressed it much, I’m assisting GitHub with these aspects across all our UI architecture.
00:30:25.320 Documentation sites, especially static content, can easily pass Axe checks. However, moving past static content becomes complex with dynamic sites.
00:30:41.520 Are we targeting AA or AAA standards? Currently, we are pursuing AA, but the goal is to achieve AAA.
00:31:00.240 Any other questions? Wow, that was a lot! Thank you, everyone!
Explore all talks recorded at RailsConf 2023
+85