00:00:04.920
Hello, my name is Joel. Thanks for coming to this session today. I'm going to share some lessons we've learned building view components at GitHub over the past year.
00:00:10.740
I'm an engineer on the Design Systems team at GitHub, and we're responsible for the Primer design system used throughout the platform. As you can see, it's really a system of systems covering areas like CSS and React. We have an icon library and Figma styles, along with the things you'd expect in a typical design system these days.
00:00:23.100
My job on the team is to make building consistent, accessible, and resilient UI in our Rails monolith an enjoyable experience, all at a pretty incredible scale. To give you an idea of our scale, here are a few stats that people always ask me about when I talk about our application.
00:00:36.420
The GitHub Rails app is over 13 years old and it receives tens of thousands of requests per second. We've extracted around two dozen services at this point into what some people call a citadel architecture.
00:00:47.100
Most of our services are written in Go and cover functional areas such as webhook delivery, but the majority of GitHub remains a Rails monolith—a big monolith that's growing quickly. We have nearly 600 models, with about 33 added in the past year, as well as almost 4,700 views, of which we've added about 20 over the last 12 months.
00:01:09.060
Additionally, we have over 800 controllers, with around 37 of those added in the last year. In total, we have over 2,000 pages in the application, and all of them need to stay current, even if they are obscure settings pages that you might never even know exist.
00:01:42.240
This scale creates some interesting problems. Things that might be a small annoyance in smaller applications can become serious roadblocks at our scale. When you have a body of work of this magnitude, patterns start to emerge.
00:01:54.000
One issue we've encountered is figuring out which template rendered a specific line of HTML. For example, when looking at a pull request (PR) page, if I want to edit the badge that displays the draft status under the PR title, how do I find the right place to make that change?
00:02:12.840
I could try searching by the specific class names on the element, but this approach is often unreliable, especially when using functional CSS class names, as we do. If we search for some of the class names on this element, we get dozens, if not hundreds, of results.
00:02:31.440
Thinking about this problem, we came up with the idea to add HTML comments at the beginning and end of each template's output, indicating the path to the template file. This way, we could see which template rendered which part of a page.
00:02:55.080
We achieved this by writing a custom ERB compiler. We defined our custom compiler as a subclass of the existing ERB compiler. That class has a method called `call` that receives a template object.
00:03:20.640
To start our method, we get the original output of the ERB compiler by calling `super`, and then we wrap that output in HTML comments with the template's path. Finally, we take that compiler and we register it with Rails.
00:03:38.940
Going back to our page, we can now see which template rendered a part of the page. In this case, it was actually our pull requests state component view component. After seeing the benefits of this patch internally, we extracted it into Rails. It is now part of Rails 6.1 and can be turned on today with a configuration variable.
00:03:55.560
It's `config.action_view.annotate_template_file_names`. For new Rails applications, this will be turned on by default in local development.
00:04:01.080
Another issue that has come up is getting our application into the correct state in local development so we can test visual changes. Part of this challenge is due to GitHub's architecture, which has a main Rails monolith and about two dozen services.
00:04:17.519
This setup makes it difficult to access the right state in our local development environment to test visual changes for our designers. Unfortunately, we don't have browser-based tests to do this, mainly because they would add too much time to our CI suite. However, we do write plenty of controller tests.
00:04:40.560
After numerous pairing sessions helping our designers get their development environment into the right state to preview the visual changes they were trying to make, I had an idea: what if we could temporarily convert controller tests into system tests? This would allow us to reuse all the existing setup code from our controller tests to preview the application in a specific state.
00:05:00.180
As a quick refresher, this is what a controller test looks like: it calls `get` with a URL and then makes an assertion—in this case, asserting against the response code. So how can we convert this into a system test? We can start by writing a module; we'll call it `SystemTestConversion`. When this module is included, it registers some configuration for Action Dispatch's system test case.
00:05:28.020
In this case, we just need to redefine `get` to visit the path provided and then pause with the debugger. For a certain response, we'll simply overwrite it as a no-op.
00:05:43.299
From there, we can conditionally include this module if `run_in_browser` is appended to the command we use to run a test. Here's what it looks like in action: what we're running here is slightly more complex than what I just showed, but it's now opening a browser, logging in, and previewing our page.
00:06:08.720
Unlike template annotations, this hasn't made it into Rails yet; I'm not sure it's as good of an idea as it served us well here. But there's a more general lesson here: our seeds and test setup code share a lot of overlap.
00:06:24.660
While working on these projects, we identified another source of friction. When modifying a template, it can be difficult to know where it's used. Not knowing where a template is used is risky, especially since we have a lot of nuanced template reuse across the application.
00:06:39.120
My colleague John made a diagram of how our templates reference each other, and navigating our render stack proved to be pretty tricky. So I built a tool called ViewFinder. Here's how it works: we pass in the template—in this case, the Wiki show page—and then extract the template string literal.
00:06:55.840
Next, we search for it in the codebase; you can think of this just like you might search in your text editor. From there, for each search result, we load the file into the parser gem, which returns an abstract syntax tree of the file.
00:07:14.880
For example, one of our search results pertains to a controller, and here's part of that syntax tree: this data structure represents how Ruby interprets the code we write. Interestingly, the Rubocop linter actually works by analyzing these data structures.
00:07:29.760
As you can see, we now have our render call to `Wiki#show`. For reference, here's the equivalent Ruby code for this tree. This is a typical respond_to block from a controller with the format HTML, calling `render :wiki_show`.
00:07:43.680
Once we have the syntax tree, we can query it. I'm not going to go into a ton of detail at this point, but we are able to find the calls to render by looking at the type of the syntax node and its method name. We do this to confirm that the render call refers to the template we're trying to trace.
00:08:00.480
We continue this process until we reach the controller action. Then, going back to the syntax tree, we look up that syntax tree until we find the definition of the controller method, which in this case is the show method on the Wiki controller.
00:08:13.680
From there, we look up the routes that render that controller action. In this case, `get 'wiki/:id'` maps to `Wiki#show`. Then we return that result to the console.
00:08:24.000
As you can see here, we have two instances of our template where the route `wiki_user_id/repository/wiki` leads to the show method of the controller and renders the show template. We also reuse that template for the index action of the controller.
00:08:37.260
So by itself, this tool is really useful; we could identify all the routes that render a template. But then we realized we could use these routes to identify which controller tests rendered our template as well.
00:08:51.420
We accomplished this through a similar static analysis process to what got us to this point. We start by finding all the `get` calls from our controller tests using this map. For example, here's a call that loads a Wiki page.
00:09:05.280
We extract the argument from this call and pass it into the Rails routing interface, which takes a request path and returns a route. In this case, it’s Rails' application routes recognizing the path. We take that path from the test and call that method, which returns a hash that includes the controller's name, its action, and the arguments extracted from the path.
00:09:22.260
We repeat this for each controller test to build a hash that looks something like this, where the controller action is the key and the matching tests are in an array of values.
00:09:40.860
With this lookup hash, we can then include which tests render the view, allowing us to use these test cases with our system test conversion tool to visually verify changes to that template in a browser.
00:09:52.920
However, there are a couple of downsides: this approach only works with explicit render calls where a template name is passed to render. This is something we enforce with a linter, but it doesn't account for conditionals. Thus, what we end up with is kind of all the possible routes that might render a template, not necessarily all the ones you want to use for a test.
00:10:05.399
This has proved really useful for us, but it’s definitely not ready for prime time. The biggest challenge we've been facing at this scale of building views is a missing abstraction in our view layer.
00:10:19.080
A common rule of abstraction is the rule of three. I really like Martin Fowler's definition from "Refactoring: Ruby Edition": the first time you do something, you just do it; the second time you do something similar, you want to reduce duplication, but you do the duplicate thing anyway; the third time you do something similar, you refactor.
00:10:39.720
Typically, when we do something repeatedly, we abstract it. If we don't, our code becomes inconsistent and hard to maintain, which is particularly true in our view code. We often build templates by copying existing ones, making sweeping changes in our view code extremely difficult.
00:10:56.399
As a member of GitHub's Design Systems team, this inconsistency really limits the leverage we have with our design system. More generally, our Rails code rarely lives up to the standards we hold our Ruby code to.
00:11:20.220
And views are generally tested with slow integration or system tests. Given all these problems, in 2019, I had the crazy idea that we could use Ruby objects to render views, inspired by ideas from React. We now call it ViewComponent, a framework for building reusable, testable, and encapsulated view components in Rails.
00:11:34.260
A ViewComponent consists of a Ruby file and a template. Here's a simple one: on the left, we have our `TestComponent.rb` file. `TestComponent` is a subclass of `ViewComponent::Base`. It has an initializer that takes a title argument and assigns that argument to the title instance variable.
00:11:46.079
In the component's template, we have a span with the title attribute wrapping an accessor called `content`, which will be the content passed into the view component for rendering. We instantiate the component, passing in the title argument and a block that says "Hello, world!".
00:12:01.380
As a result, we get this HTML: a span with the title of 'my title' and the content 'Hello, world!'. To test this component, we can write a unit test that renders the component using the `render_inline` method. You can see here we've instantiated it with the title and a block, similar to our previous example, and then we can assert against the output using Capybara matchers.
00:12:35.480
In this case, we look for a span with that specific title attribute containing the text 'Hello, world!'. These tests are really fast—in our codebase, they're around 100 times faster than controller tests.
00:12:59.220
This ability has proven to be a game changer for us. We used to treat tests that worked in the DOM as a luxury, but now they are virtually free. Since then, we rapidly adopted the pattern in GitHub.com.
00:13:17.880
Referring back to our earlier stats, the GitHub application grew by about 25% last year, except for a few components which grew by over 15 times. We’ve learned a lot in the process.
00:13:39.360
One of the goals of the Design Systems team is to help our developers build consistent UI, and view components have been a key part of accomplishing this goal. For example, last summer, my colleague Christian built a view component for the counters we use throughout the app.
00:14:00.480
You can see these counters on the repository navigation; we rendered these counters in over a hundred different places in the app. Looking at one of his pull requests rolling out this component, we can see that for one case we wrapped the integer for the counter in `number_with_delimiter`, while in another case on the same page we didn’t.
00:14:21.720
This led to visual inconsistencies, but only in edge cases we hardly would expect a developer to consider when working on a feature. Now we simply pass the raw count value to the component, which handles formatting it consistently.
00:14:47.340
Another benefit we’ve seen from view components has been in helping developers write performant code. One of the most common performance issues I've seen in view code is unintentionally querying the database. For example, we often include permission checks in our views to determine whether to display certain elements.
00:15:12.780
Unfortunately, these checks can trigger database queries if they're not batched properly. To help developers understand how their view code might interact with the database, we redefined the `render_inline` helper that we use to render view components and unit tests to accept a number of allowed queries, defaulting to zero.
00:15:29.220
So when we render the component, we fail the test if the actual number of queries does not match what was expected. This helper has proven to be quite educational; it has helped us understand the side effects of our code and prevented the introduction of several N+1 queries.
00:15:50.640
In one recent case on the checks page, this saved over 100 milliseconds of overhead per request. Another advantage we've seen with view components is how they can help manage complexity.
00:16:11.160
One example of this has been in our mailers. Building HTML emails can be tricky and often requires writing HTML like it's 1995. It's also fraught with edge cases; most of our mailers are written with that 1995 style inline HTML, which makes them incredibly hard to maintain, let alone change without breaking something.
00:16:32.399
So when we looked into adding columns to a mailer for a particular design, my colleague Mike proposed building a few components to abstract away all the tables and inline CSS necessary to implement it.
00:16:54.600
To do this, he built mailer-specific components for rows and columns, which we combined. We wrap a row containing multiple columns, allowing developers a reliable way to construct column layouts without having to deal with old-school HTML.
00:17:12.360
This is a classic case of what Rails creator DHH calls conceptual compression. By simplifying the process for building mailers, we’ve practically removed the conceptual overhead necessary to build one correctly. Instead of worrying about whether you've written your inline styles in a way that renders properly across the many different email clients, you can piece together a couple of view components and move on.
00:17:42.380
Just like ActiveRecord simplified our use of SQL, and controllers made it easy to write code to respond to requests, we use view components to make building UI correctly the default, not the exception.
00:18:04.740
Another way view components help simplify building UI is with slots. Slots are a way to pass multiple blocks of content into a single component. Let's look at an example: this is the `Box` component from the Primer design system.
00:18:24.420
It has a header, a body, multiple rows, and a footer. Here's the HTML used to construct it: we have a header that actually has a specific class you must use on a title inside the header. We have the body, multiple rows expressed as an unordered list with `li` elements using a specific class, and finally, we have a footer.
00:18:46.560
So let’s redefine this as a view component. To start, we have our app's `components/box_component.rb`. It inherits from `ViewComponent::Base` and includes the HTML we discussed. When rendering it, we instantiate `BoxComponent.new`.
00:19:05.220
Skipping a few steps ahead, you can imagine that we could create components for all these individual pieces of a box. However, this implementation places a lot of burden on developers, as the order of elements matters: the header must come first, and the footer must be last.
00:19:31.560
This implementation doesn't prevent misuse of the design system. This is where slots come into play. We can go back to our component and use the slots API to declare slots for the header, body, rows, and footer, explicitly stating that our box renders one footer, one header, one body, and potentially several rows.
00:19:50.460
In the component's template, we can refactor it to use slots instead of separate components. For example, here we render `BoxComponent.new` and then pass in a header, a body, multiple rows, and a footer.
00:20:09.960
In our component’s template, we render that header and body, and in the case of rows, they’re exposed as an array. This means we can call enumerable methods on that slot, allowing us to verify if there are any rows before outputting an ordered list. For each of those rows, we can render an `li` element with the correct class name and the content passed in for each row.
00:20:29.520
We render the footer just like we do with the header and body, all within the wrapping `Box` structure. What we've established here is a codification of our design system that aids developers in utilizing it properly.
00:20:49.020
Another tool we've experimented with is Storybook. Storybook is an open-source tool for developing components in isolation, and it's quite popular—it has around 60,000 stars on GitHub. With the help of community member John Palmer, it now supports view components.
00:21:07.440
Here’s an example of it in action with our flash view component. You can preview a change to the icon and the variant in Storybook, which gives us a development environment for building components outside of a Rails application, such as in our primary view components library.
00:21:24.960
We open-sourced this library over the summer, and it now has over two dozen components. We’re not the only ones contributing to this movement; the UK Department for Education has also released a library of more than a dozen view components for their GOV.UK design system.
00:21:44.640
One common question I receive about this project from folks outside of GitHub is: How are we rolling it out? This question is sometimes posed in a technical context regarding writing new components, refactoring old views, etc., and sometimes in an organizational context, such as how do we ensure components are used appropriately? How do we make the case for this work with non-engineering stakeholders?
00:22:04.200
Let's start with the technical aspect: how do we create view components? One way is by refactoring existing view code into components. For example, before we had view components, we used a presenter-like pattern known as view models—Ruby objects that we instantiate and then pass into views.
00:22:24.720
Here's an example view model: it inherits from a view model base class, with a status instance method that checks the repository lock method and returns either 'disabled' or 'enabled'. Here's what the test looks like: it instantiates the view and asserts against the output.
00:22:44.840
That’s a view model. We pass this object into a template and access some values, such as `view.status`. Now, here’s what it looks like rewritten as a view component: as you can see, not much has changed. We now inherit from `ViewComponent::Base`, and our class name ends with component.
00:23:09.300
The test remains similar; we now have a `render_inline` method that takes an instance of our component, and we use a Capybara matcher to assert that the text 'enabled' is rendered. The key difference is that instead of asserting against the attribute of an object like we did with the view model (`view.status`), we're asserting against what's presented to the user.
00:23:37.800
This is something we could previously only do with system or integration tests, and it’s a primary reason we refactored our view models into view components. It gives us confidence that our view code is functioning correctly from the perspective of our users.
00:23:56.460
We also write view components from scratch, considering the earlier reasons I mentioned: is there a strong need for consistency? Are there performance concerns? Is the view complex with many possible states? If so, we might opt for a component so we can unit test all the permutations.
00:24:13.680
However, the most common scenario for using view components is in place of a partial or a view model. Now, when it comes to the organizational side of things, communication has been incredibly important, taking various forms.
00:24:33.240
One way we communicate about view components is through documentation. For our open-sourced Primer view components, we use YARD, which is based on RDoc. At the top of a component's class, we include a comment describing its function.
00:24:55.440
We also provide examples written in ERB to show how to use the component, along with annotations for each initializer parameter. This strategy allows us to keep our documentation alongside our code, which helps ensure it remains up to date.
00:25:18.960
The fun part is that we leverage these annotations to generate our documentation site. In the library's Rake file, we begin by running the YARD rake task, which parses the project for YARD annotations. We then load the parsed YARD annotations into the YARD registry store object and retrieve the documentation for a specific component.
00:25:38.520
At this stage, we have the parsed documentation available in a Ruby object, which allows us to inspect it. This object includes the class definition and even has data on each method. We can look at the one marked as the constructor to see the names of all parameters.
00:25:57.420
We can also see the text of the example we wrote; unfortunately, this resides still in ERB. We can remedy this by instantiating our application controller and passing the example into the render method as an inline template.
00:26:11.460
From here, we can construct our documentation page using that data. We build a markdown file that includes the class name as the title, the component's description underneath, the inline HTML of the rendered example, and the original ERB code.
00:26:31.680
Rendered on the docs site, it looks like this. This standardized approach keeps our documentation consistent. One of my favorite aspects is that our documentation is consumable via both the component's code comments and the published site, as people often prefer one over the other.
00:26:49.320
Another way we communicate with developers about view components is through linters. I've never seen linters used as extensively as they are at GitHub. For example, we have a bot that automatically reviews pull requests, suggesting when we prefer people to use existing view components.
00:27:09.840
You can see here we have a comment indicating that we’d prefer you use the Primer Octagon component instead of this old Octagon helper, and it includes a link to our documentation site generated from our YARD comments. This saves us from having to manually make these suggestions and encode our views.
00:27:31.320
Another way we use linters is through custom Rubocop rules. We have a Rubocop rule that encourages developers to use view components instead of view models. It inspects class syntax tree nodes to see if the class is a view model. If it is, it adds an offense with a helpful message.
00:27:54.600
At our scale, if we want to implement something consistently, the best solution for us is tooling and metrics that guide developers toward the right solution automatically.
00:28:24.300
Another means of communication is through weekly updates. For many of our projects, we open an issue to track our progress weekly, sharing a few highlights, such as improvements from using the pattern. In fact, this presentation contains many examples from our weekly posts.
00:28:40.800
This practice provides company leadership with case studies they can use to justify our work. At some point, there's no substitute for investing the time, so over the summer we formed a team of about half a dozen engineers and spent two months building new view components and rolling out existing ones.
00:29:00.600
The results were quite dramatic: this graph shows the number of places we've rendered a component in the GitHub.com codebase. You can see that this graph is relatively stable until August 1st, when there is a noticeable change in the slope, reflecting the two months the team worked on the rollout.
00:29:30.600
In that two-month period, we quadrupled the number of component usages in our codebase. Even after the team disbanded, the gains continued. But how could that be?
00:29:51.060
I believe part of it is that we now possessed a critical mass of component-based examples for engineers to reference as they built new UI. But how do we guarantee that they’re using the correct examples? We accomplish this through linters.
00:30:06.780
For example, if I were to utilize the non-component implementation of the counter pattern, I would receive a message indicating that I should use the component instead. For each component, we set a limit on the number of places the non-component implementation may be used. If someone adds another usage, this linter fails their build and prompts them to use the component.
00:30:26.220
As we roll out a new component, we decrease the number of exceptions, preventing new exceptions from being introduced. We also reuse this code to generate a report on our progress. For instance, we have successfully migrated every single blank slate in GitHub.com to use the view component we built.
00:30:45.060
Another question I often receive about this project is: what have we learned from open sourcing it? I should start this section by noting that this is the first open source project I've worked on, but by many standards, the ViewComponent project has seen considerable success.
00:31:05.520
We’ve been fortunate to receive a lot of engagement on the project, with code contributions from over 100 developers, of whom only about a dozen work for GitHub. Collaborating with so many people from diverse backgrounds has taught me invaluable lessons in empathy. What has stood out to me is that every issue, PR, or discussion post has value.
00:31:30.840
Even small missteps, like someone being confused by the documentation and thus writing a component incorrectly, are evidence we can utilize to enhance our docs. If an error message doesn't aid someone in resolving the issue and they file a report, we should probably reconsider that error message.
00:31:49.560
It's clear to me that people want to contribute, but they aren’t always sure how—that's why we focus on making it easy for people to contribute to the project. We want to empower people to make quality contributions with minimal effort.
00:32:10.260
The library isn’t particularly complex, but it does have nuances and varying behavior across Rails versions, which adds a critical layer of complexity. One approach we have used to ease this burden is matrix builds.
00:32:27.600
By running the test suite across a dozen combinations of Ruby and Rails, we ensure that every change works across all versions, and even more importantly, we guarantee 100% test coverage by combining coverage data from all these builds.
00:32:49.620
This assures us that the contributions from the community are reliable and that they won’t disrupt existing APIs. Another lesson I've gleaned from working on this project is the immense power of Rails conventions.
00:33:06.180
When writing view components, we strive to conceptually align the framework with Rails as closely as possible. What we've seen is that Rails conventions create strong baseline expectations, even outside of Rails itself.
00:33:24.120
These conventions have enabled others to instinctively know how to contribute to the project. For instance, Juan Manuel contributed view component previews based largely on conventions established by Action Mailer previews.
00:33:39.780
I think it’s worth discussing what has not been going smoothly in the project. There's one problem I can't shake: we now have two distinct methods of writing views that operate differently.
00:33:56.580
I can't envision Rails incorporating native support for view components as they currently exist, in a separate directory. However, it makes me ponder whether we could extract lessons from view components into Rails.
00:34:12.420
One example is Andrew Culver and Dom Christie's nice_partials gem, which offers a similar API to view component's slots feature, allowing multiple blocks of content to be passed into a partial. Perhaps this is something we could integrate into Rails.
00:34:31.140
I believe this is the long-term focus we need: from our work on view components, we can identify opportunities to improve Action View. As the view components API matures, it will clarify the potential enhancements.
00:34:50.520
By doing so, we will shape the future of UI building in Rails based on experience rather than hypothesis. An example of this is the lack of robust support for view caching in view components, which we don’t use extensively at GitHub, making it hard for us to build framework support for it.
00:35:09.420
This presents one area where we could utilize your help. More broadly, this project has pushed me to contemplate innovation within Rails.
00:35:20.760
There’s a book titled "The Innovator's Dilemma," which is widely recognized as one of the most influential business books ever written. It addresses the difficult choice between meeting current needs or embracing new innovations that could address future needs.
00:35:39.360
The book illustrates how successful organizations can do everything right yet still lose their market leadership or even fail as unexpected new competitors emerge and capture the market.
00:35:56.640
I believe this lesson is crucial as we consider Rails's long-term relevance. This is especially important to me because GitHub's future is heavily reliant on Rails remaining relevant.
00:36:12.960
Given that we have so much built on top of Rails, it’s very unlikely we would rewrite our core application. So what does all this mean for us?
00:36:29.520
We need to innovate and experiment with potentially disruptive ideas, such as view components. Improving Rails remains our only path to survival. We need to actively participate in maintaining Rails's relevance for the long term.
00:36:51.060
As good citizens, we should contribute to the framework. Rails was built by people just like all of you, and we benefit greatly from the work others have done. However, we need more voices.
00:37:06.600
We need more extractors. Many Rails developers have worked on multiple applications; we know the underlying deficiencies and the gems we tend to integrate into every app. We understand the pain points.
00:37:23.220
The survival of our companies depends on it because choosing a framework has lasting implications. Rewrites are rare, and yet the world moves on. Thus, it is our responsibility to keep Rails relevant.
00:37:42.300
Thank you.