Encapsulating Views

Talks

Encapsulating Views

Joel Hawksley

Encapsulating Views

by Joel Hawksley

In the video "Encapsulating Views," Joel Hawksley discusses the challenges of Rails views, which lack encapsulation, making them difficult to test and reason about. As a software engineer on the Design Systems team at GitHub, Hawksley presents a solution called ViewComponent, which offers a structured way to encapsulate view logic and enable better testing.

Key Points Discussed:

- Encapsulation in Rails Views: Unlike models and controllers, Rails views do not encapsulate data or methods effectively, leading to complexities and testing difficulties.

- Common Abstractions: To manage complexity, the Rails community has developed several abstractions like decorators, presenters, and React components. However, many developers resort to combining these techniques, which complicates the view structure further.

- Testing Challenges: Rails does not provide built-in support for unit testing views, forcing developers to rely on more expensive UI tests. This results in slower test execution and increased maintenance overhead.

- Introduction of ViewComponent: The ViewComponent pattern creates encapsulation at the view level, promoting a cleaner structure. Each component is encapsulated, meaning they don’t share global state, making them easier to test and maintain.

- Community Contributions: Since its release, ViewComponent has seen vast community support, leading to numerous contributions that improved its functionality, such as adding generator support and introducing content areas for organizing layouts effectively.

- Performance Benefits: The architecture of ViewComponent significantly reduces runtime compilation overhead by compiling views at application startup, which enhances performance during the first request handling.

- Future of Views: Hawksley emphasizes a pivot towards utilizing components for all new views within applications, suggesting that components can effectively replace partials, improving encapsulation and consistency across the application.

The main takeaway from Hawksley's presentation is that adopting a component-based architecture in Rails allows developers to create more maintainable, testable, and performant view layers. Ultimately, this fosters better development practices and enhances user experience by providing a more robust and organized codebase.

00:00:08.910 Hello there, welcome to RailsConf 2020, Couch Edition. My name is Joel Hawksley, and I am coming to you from my closet. This is the best place I could find in my house to make a nice clean audio recording. I'm an engineer on the Design Systems team at GitHub, where we are responsible for the Primer design system used throughout all the applications GitHub builds.

00:00:23.380 Today, I'm going to talk about a project I've been working on for the past year called ViewComponent. I will cover why we built it, how we're using it, some lessons we've learned, and how you can get involved in the future of the project.

00:00:35.829 But first, a little about me: I live south of Boulder, Colorado, with my wife Caitlin and our wonderful 90-pound golden retriever, Captain. Before becoming a software engineer, I was a sports photographer for a couple of years. I traveled the country shooting every sport imaginable and photographing portraits for marketing campaigns.

00:00:52.449 This is a portrait of a basketball player for a billboard. I really enjoyed covering the wide variety of sports. Here’s an image from a NASCAR race in Martinsville, Virginia. I also did a lot of work covering daily life—one of my favorite photographs was of the first day the swimming pools opened during an especially hot summer in Grand Rapids, Michigan. I covered breaking news too, including scenes from a car accident in southern Illinois.

00:01:16.780 I did that for a while until 2013, when I was out riding my bike on the Blue Ridge Parkway and received a call from my photo editor. She told me she had been laid off, which was a tough day for me because I had aspired to have her job. In that moment, I saw the future of my career vanish before me.

00:01:31.600 That same day, I called my friend Erin and asked for his advice. Not only did he give me advice, but he also offered me an apprenticeship. I moved to Rhode Island, and after a couple of months, he taught me how to be a professional software engineer. I truly wouldn’t be here if it wasn't for him—thank you, Erin.

00:01:53.040 Enough about me. It’s an especially stressful time for many of us right now. I'm recording this talk in my closet because we are all worried about the pandemic. While working on this talk, I spoke to my therapist, who emphasized that it is important for us to focus on gratitude during times like these.

00:02:05.770 So I want to start my presentation with some gratitude. I am thankful for Rails. Rails has been around for a long time as a programming framework, and we should appreciate how much Rails has enabled all of us to accomplish. We must also be grateful for the value Rails has brought to the world. It has allowed us to focus on the needs of the people who use our products.

00:02:25.000 Rails has also been integral to GitHub's success. More than 800 people contributed to our Rails monolith last year. Rails has scaled with GitHub; our application is essentially just vanilla Rails, but with significant scale. We have over 700 models, more than 600 controllers, and as of yesterday, over 4,400 views.

00:02:49.150 It is this specific aspect of our scale that I will focus on today. As Rails applications grow, we often turn to view abstractions not provided by Rails. Here are a few examples based on my experience.

00:03:04.660 One abstraction we turned to is decorators. I encountered this at my first job at a little consultancy called Mojo Tech. After working on a couple of Rails projects there, I landed on my first large-scale Rails application, where I first saw a folder called 'decorators' that felt somewhat out of place. It had model view logic that looked something like this.

00:03:19.419 Here we have a user decorator class with a status instance method that checks if an active predicate is true. If true, it returns one string, otherwise it returns another. These decorators served as a place to put view-related logic coupled to a specific model.

00:03:39.990 However, in my experience, we found that this was just another way to hide methods we felt uncomfortable placing on the model itself. These decorator objects were unit tested like models. For example, here's a mini-test for the same decorator: we set it up with a user and then call an instance method to assert that it returns the inactive string.

00:03:59.800 Another abstraction I came across was React on Rails, which is a plugin for Rails that allows you to render React components very easily. I used this at a startup where we utilized React to build our user interface. However, it was interesting to see how it began to encroach on areas of our application where traditional Rails templates could have sufficed.

00:04:06.420 We were using React because it enabled us to write unit tests for our views. In this case, the tests were quite simple: for example, if we had a button component, we would simply render the component and assert against the HTML result. These tests were very fast, allowing us to write a lot of them.

00:04:40.110 A third abstraction we used was presenters, which in our application at GitHub, we refer to as view models. They encapsulate view-specific logic extracted into Ruby objects. For example, we have a presenter for the repository index view with a status instance method that returns either 'disabled' or 'enabled' based on the repository status.

00:05:02.330 When we test it, we can do so just like any other Ruby object and check that the status instance method returns 'enabled'. However, in my experience and in our codebase, these presenters ultimately became a distraction. They gave us a false sense of confidence; we thought we were testing a view layer, but we were really just testing an object designed to build our views.

00:05:27.370 Another pattern I’ve observed for handling complexity in views is adding more logic to partials. This is something where we particularly fall short. Our templates often contain a lot of Ruby code within them, which simplifies some logic but makes it difficult to test. For example, we could translate the previous view model example into inline Ruby.

00:05:50.370 We can assign the result of a ternary operator to a status variable and render it in an h2 tag. However, in practice, this situation can worsen significantly, resulting in files that contain Ruby blocks extending dozens of lines long. The issue is that this complex logic cannot be tested directly, as Rails does not provide a straightforward method for unit testing views.

00:06:04.740 Reflecting on my experience, I wondered if my observations were a reflection of the broader Rails ecosystem or merely a reflection of the applications I had encountered. To investigate, I conducted a survey among my local Ruby group in Boulder, which consists of about 50 people. I asked them to share screenshots of their application folders, and here’s what I found: some respondents used decorators and presenters, but often applications utilized more than one abstraction, and in more than half the responses, people reported using view-related abstractions not provided by Rails.

00:06:46.550 This trend is prevalent across the Rails ecosystem. There are also some popular gems for these patterns, such as Draper and Cells, which collectively have millions of downloads. This indicates that something is clearly missing; we are turning to these abstractions and breaking from Rails convention.

00:07:06.050 In conversing further with people who responded to the survey, it became evident that the primary motivation was centered around testing. One of my favorite frameworks for discussing testing is Martin Fowler's test pyramid. It’s an effective illustration of the trade-offs associated with various test types. The pyramid emphasizes that we should be writing as few tests at the top, which are slow and expensive, and more tests at the bottom, which are fast and cost-effective.

00:07:42.330 In the context of Rails, we have UI tests, comparable to system tests, and service tests that can be seen as controller tests. However, Rails does not provide a way to write unit tests for views, thereby limiting us to the most expensive options.

00:08:00.400 As our applications grow, we often resort to view objects, partly because they can be unit tested, thus allowing for more thorough coverage of our view layer without the high cost associated with controller or system tests.

00:08:25.310 But even if our views could be unit tested, my exploration of this problem suggests a more significant issue: encapsulation.

00:08:49.810 So, what is encapsulation? According to Wikipedia, encapsulation in object-oriented programming refers to bundling data with methods that operate on that data or restricting direct access to certain components of an object. I may not consider myself a computer scientist, but this description resonates with my understanding of object-oriented programming.

00:09:06.050 Rails does provide encapsulation at times; for example, models are often encapsulated quite well. If we consider a user model, a named instance method provides public access while a callback method, such as for sending a welcome email, acts as a private method, restricting direct access from outside.

00:09:25.600 The testing of the user model occurs against that public interface, not the private methods. As for controllers, they also exhibit some encapsulation. For instance, in a user's controller, the show method serves as a way to access user data while the private current user method restricts direct access.

00:09:41.160 Even when testing, we interact slightly indirectly with the show method and make assertions based on the outcome of invoking the private current user method without calling it directly.

00:09:54.920 So, what about views? The closest we come to encapsulation in views is local variable assignments within the template. However, when it comes to testing, we recognize that our views lack a public interface, which complicates unit testing.

00:10:12.720 To understand why encapsulation of views poses an issue, we first need to comprehend how views function. Unlike models and controllers, views are not objects. Let’s take a closer look.

00:10:35.330 We'll examine the explicit rendering of a partial from another file. For illustration, consider the following example code: in the first view, app/views/demo/index.html.erb assigns an instance variable called message with the value 'Hello, world' and then renders a partial named message.

00:10:55.960 In our message partial, app/views/demo/message.html.erb, we have an h1 tag that renders the value of the instance variable message. Initially, I did not expect this to work, but interestingly, when we render the index page, the second view can access the instance variable from the first.

00:11:11.950 This made me curious about what happens when the render method is invoked. It turns out a lot happens, in fact, it’s one of the more beautifully engineered aspects of Rails, though it's so complex that I can't discuss it all today.

00:11:30.760 Let’s skip several steps down the call stack to the action view template render method. One intriguing aspect of this method is that it calls another method called compile with a bang. Now we find ourselves in Ruby land, where the use of the term 'compile' in a dynamic language raises questions.

00:11:55.990 Looking at the compile method, there's an informative comment that reads 'compile a template'. This method ensures a template is compiled just once and removes the source after it has been compiled, wrapping the non-bang compiled method by passing in a variable called mod, which refers to the view's compiled method container.

00:12:17.750 If we explore that non-bang compile method, we encounter an intriguing line: code equals handler.call(self.source). To understand this context, let’s consider what handlers are. The handler is defined in action view template handlers ARB, which, for the sake of this exploration, serves as a tool to work with ERB.

00:12:29.490 Self refers to an instance of action view template that includes the path to the template and the source of that template. When we call this line, it yields a string that is essentially the compiled version of our ARB template. Breaking it down into multiple lines illustrates that handler.call turns our ERB into Ruby.

00:12:46.780 In effect, we've compiled our template into Ruby code, allowing us to define a method. Seems straightforward enough; however, the method name is concatenated with a path and employs virtual path collision prevention, generating a name for our template.

00:13:01.550 The compiled version of our template ultimately leads to a callable method. Returning to the action view template, we evaluate this method against the action view base's compiled method container. At the end of this process, we have a method corresponding to each view in our application.

00:13:16.430 We see how our index template, our message partial, and additionally our application layout become methods available to the action view base. Consequently, the rendering process accomplishes the compilation of our templates into executable methods.

00:13:30.030 When we invoke the index action and render the index template, we assign the instance variable message the value 'Hello, world.' Consequently, when we then render the message partial, we merely call another instance method on the action view base, where access to the message instance variable persists.

00:13:51.490 This global context across views, helpers, and components incentivizes shared state, which stands in stark contrast to the principle of encapsulation. In our application, we have over 4,400 view files all executing in this shared scope.

00:14:04.290 The implications are significant; it complicates debugging, as the global context makes it difficult to reason about execution behavior. Moreover, it hampers unit testing as views cannot be isolated, which is problematic because views hold considerable business value.

00:14:18.430 Ultimately, views represent what we deliver to our users, yet these complications mean they are frequently a blind spot in our architecture. Given this situation, I started contemplating how encapsulated views could look.

00:14:34.510 We are aware decorators and presenters are attempts to introduce encapsulation into the view layer. These are solutions designed to facilitate testing. Throughout 2019, I worked on a vision for addressing these encapsulation issues, which led to the creation of ViewComponent.

00:14:49.390 First and foremost, ViewComponents aim for encapsulation by ensuring each view has its own context, rather than sharing a single context across multiple views. In this example, we have a simple message component.

00:15:01.910 In our app components directory, the message component defined in Ruby accepts a single named argument, assigning it to an instance variable called message. The component is accompanied by a message component template, which contains the same ERB markup that we previously used in a partial.

00:15:24.290 When we render this component, we simply pass an instance of that component object into our app/views/demo/index file. This direct referencing simplifies usage, eliminating ambiguity regarding the relationship between a string and a template in our views.

00:15:39.680 If we wanted to locate each instance of the message component utilized across our codebase, we could simply search for the class name, making maintenance more straightforward.

00:15:53.490 In terms of compilation, we follow a similar process as we examined earlier, but instead of assigning the compiled view to a global context, we directly attach it to the component object.

00:16:05.810 Thus, when rendering this template, it only has access to its own state. This isolation alleviates concerns about instance variables leaking across different contexts. It also facilitates encapsulation of helper access.

00:16:20.600 In this case, to reference the star icon helper within the template, it’s necessary to include that helper within the component. We can also utilize a helpers dot escape hatch that resembles how Rails enables access to view helpers in controllers.

00:16:39.080 Unit testing components becomes straightforward. We can render the component inline using a render inline helper and then assert against the rendered result using Capybara matchers, which are the same matchers utilized in Rails system tests.

00:16:54.200 Notably, this approach does not involve a browser running in the background, allowing these tests to execute much faster. We launched this library as a gem in August, extracting it from the GitHub monolith.

00:17:09.770 This represented my first venture into open-source projects, and we have been amazed by the support we have received. As of mid-April, we have shipped 39 releases, including our second major release of version 2.0 in late March.

00:17:24.170 Most of this work has stemmed from the community; we've had over three dozen contributors from across the globe. Some notable contributions include simple things like generator support, contributed by Nathan Stock at Shopify.

00:17:39.100 More complex features have also been added, like component previews, contributed by Manuel Maya from Argentina. These function similarly to mailer previews: a component preview inherits from a preview class and defines methods for each preview, generating a user interface for clicking between states.

00:17:54.350 Another excellent feature contributed by John Palmer from Boston was the addition of content areas. To explain this feature, let’s review what a box component from our Primer design system looks like in HTML. The box includes a header, body, and footer wrapped in a div.

00:18:09.600 Returning to our HTML example, we can turn it into a view component by crafting a box component and its corresponding template, ensuring the structure closely resembles what we presented previously. In our index action, we now render this component by passing an instance of it.

00:18:28.140 Skipping a few steps, it is possible to create several components: one for the header, one for the body, and one for the footer. This allows a single component to handle the overall structure while rendering each piece accordingly.

00:18:44.390 While this architecture is useful, it does not guard against misuse of the design system. The header should always be first, and similar constraints apply to the footer, which should always be rendered last.

00:19:00.800 This is where the content areas feature comes into play. By declaring a content area for the header, body, and footer, we can refactor the usage of what would effectively be four components into a single component, utilizing named blocks instead of separate components.

00:19:18.080 This enables us to pass named header, body, and footer blocks into the component template, allowing for clearly defined rendering within the wrapping div. The beauty of this solution is the enforcement of design system conventions, ensuring correct rendering of elements, regardless of the order in which named blocks are passed.

00:19:30.280 Furthermore, since components are merely Ruby classes, we can extract them into gems. While we have not yet done this, there are examples of companies in the financial sector that have multiple Rails applications using a single gem that encapsulates their entire design system.

00:19:45.530 As a final note, it’s particularly exciting that we have integrated support for third-party component libraries, like ViewComponent, natively within Rails. This feature will be released in Rails 6.1.

00:20:03.790 Now, addressing performance: the current implementation of Rails views can lead to inefficient caching behavior, especially at scale. Views are compiled into methods during runtime after server processes, such as Unicorn and Puma, are initiated. Due to the high load, this can often lead to longer initial render times.

00:20:23.040 This compilation takes place in every server process each time a view is served for the first time. We encounter a 'cold first request' render time that often proves higher than necessary. However, with ViewComponents, we are able to circumvent this issue.

00:20:42.460 We define an initializer that compiles all descendants of the ViewComponent base class, ensuring that templates are prepared ahead of time. For instance, the template of our example component compiles into a callable method.

00:21:00.080 This preparation occurs at application startup, before worker processes are spawned. This yields significant performance improvements during testing.

00:21:18.910 In a comparison with typical tests from the GitHub codebase, I found that a standard component test typically takes about 20 milliseconds, while one of the simplest controller tests could take 1.5 seconds—73 times slower.

00:21:38.050 Notably, this discrepancy is conservative; the common difference sits around 100 times or two orders of magnitude. In a real-world example, testing our issue state component with six unit tests takes around 120 milliseconds, as opposed to nine seconds if they were structured as controller tests.

00:21:56.540 The implications are significant since we have over 4400 views in our monolith, with over 3000 being partials. Without unit tests, we would likely end up testing them in various controller tests across the application, adding to performance issues.

00:22:11.750 Utilizing view components to write unit tests allows us to thoroughly test our view layer permutations at the unit level without duplicating tests. This approach permits us to reserve more expensive service and UI tests for essential happy path assertions.

00:22:27.260 Throughout the past year of experimentation with this pattern, we have learned many lessons, some of which were not favorable. One significant mistake we encountered pertained to using validations.

00:22:42.440 For instance, we have an avatar component defined with a default size constant and an allowed sizes constant that includes these values. We thought it prudent to implement active model validations to verify that provided sizes fall within the established range.

00:23:06.140 While this approach fulfilled initial expectations, it resulted in complications during deployment, causing validation errors not foreseen in our tests. This led to a scenario where the application inadvertently failed for a handful of users.

00:23:24.060 We swiftly addressed the issue with a rollback, but it made us rethink how components should handle invalid input. Our solution evolved into a simple helper function, fetch or fallback.

00:23:43.780 The fetch or fallback method accepts three parameters: an array of allowed values, a given value, and a fallback. It checks whether the provided value is in the allowed list—if it is, it returns that value; if it isn’t, the method returns the fallback instead.

00:24:14.460 However, if we are in the development environment, it raises a helpful error message. This enables us to maintain the desired feedback from validations while safeguarding against production errors.

00:24:37.280 Consequently, when we refer to the avatar component's size attribute, we use fetch or fallback to ensure the value aligns with the expectations we’ve set. The lesson gained here is fundamentally about designing with resiliency in mind.

00:24:53.790 We underestimated the downstream risks posed by validations within our components.

00:25:05.770 More broadly, we learned that eliminating error cases from our systems is of paramount importance.

00:25:29.380 Another insight emerged from our experience with component refactoring: it can often unveil complexities we weren't previously aware of. One of the more intricate cases involves a details dialog partial, which is used around 220 times across approximately 150 files and handles a significant amount of complexity.

00:25:46.510 Refactoring this complex dialog into a component illuminated its intricacies. The initializer required 17 potential arguments, heightening awareness of this complexity and prompting us to tackle it more systematically.

00:26:05.700 This newfound visibility allows us to ensure comprehensive unit testing for these components. In this case, we managed about 20 cases, but it could likely benefit from further testing.

00:26:21.060 Moreover, as we uncover complexities, awareness increases, and we can work towards simplifying the components.

00:26:39.410 We've also noted the necessity of maintaining consistency. This is best illustrated through the implementation of the PR state component, which we utilize in nine different areas of our application.

00:26:58.960 In the process of implementing the component, we observed several instances of similar code being copied and pasted across implementations. This revelation indicated that multiple usage scenarios had not been updated to accommodate draft pull requests, recently added functionality.

00:27:12.500 By standardizing through the component, we ensured consistent handling of these various cases, delivering a high-quality user experience.

00:27:30.110 Additionally, we discovered the advantages of composition. For instance, the PR state component wraps another component from our design system, rendering the primer state component based on arguments mapped from a pull request object.

00:27:51.580 This pattern of application-specific components taking, say, an ActiveRecord object and translating it into a design system component has become common as we develop components in our codebase.

00:28:06.700 One great example is the primer layout component created by my colleague John Rohan. The component organizes rendering of a simple layout with a main content body and a sidebar.

00:28:21.510 This layout is employed in several action pages, such as pull request details, where the main content and sidebar are rendered together, incorporating elements like labels and reviewers.

00:28:38.260 The initializer for this component takes several arguments, including an on/off switch for responsiveness and the intended sidebar location, giving it a default setting of right.

00:28:55.500 We validate that the sidebar can only be set to either left or right and use fetch or fallback to manage the argument.

00:29:06.450 The component also leverages the multiple content area feature to capture the main and sidebar content, rendering them side by side with responsive design considerations.

00:29:19.740 The realization of this implementation highlights the complexity of achieving responsive behavior correctly. Now, we possess a single version of this pattern that is utilized across several places in the app.

00:29:36.680 What does this all mean for the future of our views? We are starting to migrate our view models to components, and we’ve implemented a linter to restrict usage of view models and encourage usage of components instead.

00:29:58.950 As of mid-April, we have created 112 components within our application and an additional 16 in our design system, with usage spanning around 250 views.

00:30:14.170 When considering new views, we’ve generally followed this rule: if it could be a partial, it can be a component.

00:30:28.450 We have open-sourced the Ruby library on GitHub since August, and contributions are welcome. We would love to see you all try it in your applications and provide feedback. Thank you!

RailsConf 2020 CE