Sorbet at Grailed: Typing a Large Rails Codebase to Ship with Confidence

Summarized using AI

Sorbet at Grailed: Typing a Large Rails Codebase to Ship with Confidence

Jose Rosello • November 08, 2021 • Denver, CO • Talk

In this talk presented at RubyConf 2021, Jose Rosello, a staff engineer at Grailed, discusses the integration of Sorbet, a gradual static typing system for Ruby, into their extensive Rails codebase. With Grailed being a peer-to-peer luxury fashion marketplace with over 7 million users and a growing codebase from 30,000 lines to over 150,000 lines in just a few years, the need for stronger type safety became apparent. The presentation highlights several critical points regarding the transition from dynamic to static typing:

  • Context and Motivation: Grailed's transition to static typing was largely driven by a developer's frustration with null pointer exceptions and type issues in a dynamically typed language.

  • Benefits of Static Typing: Static typing allows for more confident refactoring and reduced errors, improving both code legibility and documentation reliability. The reliance on comments for documenting types can lead to stale and misleading information.

  • Introduction to Sorbet: Sorbet allows developers to gradually adopt static typing through its flexible enforcement levels (false, true, strict, strong). Rosello emphasizes that the syntax allows for valid Ruby code without a transpilation step.

  • Runtime Checks: Unlike traditional static typing, Sorbet provides runtime checks to catch type errors that arise from dynamically typed code calling typed methods, reducing the likelihood of production errors.

  • Handling Dependencies: Sorbet utilizes Ruby Interface Files (RBIs) to enable type checks for external libraries and gems, promoting speed and practicality without needing to alter the original gem code.

  • Rolling Out Sorbet: The adoption process included generating RBIs, addressing dynamic imports, and utilizing tools like Tapioca for better RBIs generation. Key metrics show an impressive uptake, with only 8% of code files remaining untyped.

  • Challenges: Rosello discusses challenges such as ensuring types for dynamic Rails components, dealing with legacy code, and managing a gradual typing approach. He acknowledges the limitations of Sorbet and encourages the community to embrace its continued development for improvement.

  • Conclusion: After two years, Sorbet has greatly enhanced code quality at Grailed, promoting clearer documentation, safer code changes, and overall better developer experience. Jose concludes by affirming the positive impact of Sorbet and mentions that Grailed is hiring, inviting interested candidates to learn more.

Overall, this talk combines practical insights with technical analysis, demonstrating the substantial benefits of incorporating static typing into dynamically typed languages like Ruby.

Sorbet at Grailed: Typing a Large Rails Codebase to Ship with Confidence
Jose Rosello • November 08, 2021 • Denver, CO • Talk

Called “downcase” on nil? Forgot to return the right object in one of your logic branches? Called “first” on a String instead of an Array and spent half an hour trying to figure out why a single character was getting passed around everywhere?

At Grailed, these situations were not uncommon. We are the largest marketplace for luxury men's fashion, with over 7 million users, and a growing Rails codebase that spans hundreds of thousands of lines. Before typing, changes to core interfaces often required creative grepping, modification of type checking unit tests, and updating brittle type documentation.

Ever since we started gradually typing our codebase with Sorbet, we’ve been able to make intrusive changes faster and confidently. In this talk, we’ll walk you through our prior art, challenges, learnings, and big benefits of typing our codebase.

RubyConf 2021

00:00:10.240 Thank you, everybody, for coming to this talk. I know there are at least ten other talks happening right now.
00:00:16.160 All of them seem really cool, so I appreciate you coming to my talk. I'm going to get into Sorbet, which is a gradual static typing library for Ruby.
00:00:29.359 You've probably heard of it, and specifically, I'm going to talk about how we use it at Grailed to gain more confidence in the code that we're writing and shipping.
00:00:37.360 First, a little bit about me: my name is Jose Rosello, I live in Brooklyn, and I am a staff engineer at Grailed.
00:00:49.039 Over the past couple of years, I've worked on infrastructure and security, and the last thing I'm doing is payments. I've worn a few different hats during my time here.
00:00:55.760 This is an overview of the talk. I'll give you some context about where Grailed was and what finally motivated us to adopt static typing.
00:01:07.840 I’ll provide a quick crash course on Sorbet and walk you through how we rolled it out at Grailed, as well as some of the challenges we faced. Then, I’ll leave you with some food for thought.
00:01:21.200 Grailed is a peer-to-peer marketplace focused on style and fashion, primarily luxury goods for men. Think of it as eBay, but if eBay were cool.
00:01:32.240 We currently have over seven million users, and we use Rails almost exclusively to power our API. It's a large Rails monolith that has grown from 30,000 lines of code in 2017 to around 150,000 lines today.
00:01:45.840 Our developers are fairly familiar with static typing and gradual typing through our front end; we use TypeScript there for our Next.js code. The last line I have here is a bit of an exaggeration, but it truly was motivated by an engineer who was tired of shipping null pointer exceptions and subtle type issues.
00:02:10.800 He figured this would be a good exploration to see if it could solve those problems. So, why static typing in Ruby, to begin with?
00:02:18.239 For me, the biggest concern was feeling confident around refactoring and knowing the changes you make are safe. Ruby is dynamically typed, so if you want to refactor a method, you might get lucky and find it easily, but what if the method name is common?
00:02:43.360 Then you need to be really good at grepping and inspecting call sites. You might think unit tests will catch any changes, but they are subject to human error and the quality of the tests themselves.
00:03:01.599 We love our unit tests and have a lot of good coverage, but you can still miss things, especially with logic issues not covered by new tests.
00:03:14.560 Rails and Active Record are highly dynamic, making it easy to make assumptions about data presence. This can lead to null pointers sneaking in and breaking your code. There are numerous instances of type issues in the form of pull request titles that we've fixed over the years.
00:03:43.440 The other side of the coin is legibility and documentation. Using YARD docs has its advantages as they provide a standardized way to document types, but comments are just that—comments. Developers are not generally trained to assure comment accuracy. When code is refactored, it's easy to forget to update comments, leading to stale or misleading documentation.
00:04:41.199 Working at Grailed, which is now about 150,000 lines of Ruby code, you're often faced with unfamiliar code. Seeing types that are enforced by the type checker provides a clearer understanding of what the code does.
00:05:22.320 Before I dive into Sorbet, I acknowledge that this is a Ruby conference, and discussions on typing can evoke strong feelings. Some people don't see the value in type annotations.
00:05:35.840 I looked for scientific research supporting types, but the results were fairly split. It's unclear if the trade-offs of adding static types are worthwhile, and much of the research focuses on toy codebases in older languages without modern type system guarantees.
00:06:06.960 That being said, I'll get into Sorbet. If you dislike strong types, please don’t heckle me; just enjoy the presentation.
00:06:32.000 Sorbet is a gradual typing library. This means you can choose your own level of enforcement, and regardless of your codebase size, you can get started with Sorbet almost immediately by implementing minimal enforcement.
00:06:49.679 Sorbet does this through the concept of sigils—magic comments declared at the top of files. Enforcement happens at the file level. If you don't have a sigil, Sorbet treats your file as if it has the default setting, checking only for syntax errors and missing constants.
00:07:19.679 With 'true' sigil, Sorbet checks that invoked methods exist and verifies the correct number of arguments are passed. Any statically typed signatures in your file will also be validated.
00:07:43.440 The 'strict' sigil means every method and constant must have static types declared. Lastly, 'strong' is the ultimate level of enforcement, requiring that every method call invokes typed code.
00:08:06.879 We aim for 'true' on existing files, and for any new code, we strive to make it 'strict.'
00:08:37.200 Here's an example of a Sorbet signature. If this is your first time seeing one, you might find it looks a bit hideous compared to other programming languages. The key takeaway is that this is still valid Ruby code.
00:09:18.560 Unlike languages requiring transpilation, Sorbet allows you to ship valid Ruby directly to production. This functionality also enables Sorbet to perform runtime checks.
00:09:27.760 I believe this is really crucial, especially for gradual typing systems. Since some code will be untyped while other code is typed, the untyped code may invoke methods that are typed.
00:10:01.360 At runtime, if Sorbet detects that the wrong types are being passed to a typed method, it raises an exception. However, at Grailed, we treat these alerts differently.
00:10:32.640 Instead of treating these as critical failures, we alert ourselves through a Slack channel. When we add types to old code and the wrong inputs are detected, we receive a Slack notification, allowing us to correct the issue without punishing ourselves in production.
00:11:06.720 This process has been beneficial; we've learned a lot about our codebase, added types in the process, and we’ve never broken production due to this.
00:11:37.200 Although we've primarily focused on typing our code, what about our dependencies? Sorbet's answer to this is the Ruby Interface file (RBI).
00:12:11.040 An RBI essentially serves as a skeleton of code, declaring classes, methods, and constants without actual body implementations. Sorbet inspects your gem code and generates these declarations.
00:12:43.040 These skeletons have advantages; firstly, they allow Sorbet to maintain speed without having to inspect entire gem bodies, which could slow down performance. Secondly, if type inconsistencies arise in your gems, you may need to manually edit the incompatible code.
00:13:18.159 Let me walk you through an example of what an RBI looks like when generated by Sorbet after inspecting a gem. It outlines the general structure of a class but does not provide information about types.
00:13:45.600 In another example, we have a designer, which is an Active Record model. This one was generated by Sorbet Rails, which inspects Active Record models and generates signatures based on the database column types.
00:14:02.400 Because Active Record understands the types of database columns, it can generate more informative signatures. This demonstrates Sorbet's capability of understanding more complex types.
00:14:35.360 Next, I'll cover some of Sorbet’s greatest features. One is T.nilable, which functions like an option type found in other programming languages, ensuring that you can't call a method that might not belong to your object.
00:15:13.920 Another feature is exhaustiveness checking. If you create an enum but fail to account for all possible values in the calling code, Sorbet will alert you of unhandled cases.
00:15:32.640 The type checker will let you know if any potential values are overlooked, which is particularly useful for errors in payments, where such considerations are crucial.
00:16:14.560 T.struct is another utility we frequently utilize. It allows you to declare fields and types for a data structure while optionally enforcing immutability, making it clearer to understand what kind of data is being handled.
00:16:52.480 An issue with Ruby is its implicit returns, which can lead to unintended consequences. The void return type prevents this and guarantees that a method behaves correctly without relying on output.
00:17:34.560 Now, let me walk you through our rollout process. The initial steps are fairly straightforward, and Sorbet comes with utilities that help generate RBIs for your gems, allowing you to start running checks almost immediately.
00:18:05.920 We encountered issues with dynamic imports that led to undefined constants identified right from the start. This was particularly challenging due to Rails’ highly dynamic nature.
00:18:37.679 We leveraged Sorbet Rails, which assists in addressing many Rails-related challenges by providing RBIs for all dynamic methods generated by Rails, particularly for Active Record models.
00:19:06.760 We also utilized Tapioca, a tool from Shopify that provides similar functionality but is friendlier and offers more customization options. For the Ruby linter, we use RuboCop, which has an add-on for enforcing sigils on top of files.
00:19:46.480 Moreover, Shopify provides a separate tool called Spoon that automatically increases the strictness of files when it's confident that no issues exist.
00:20:12.320 After setting up the initial steps, the main focus for us became evangelizing Sorbet and educating team members. Thankfully, many developers were curious and eager to adopt types.
00:20:57.920 We held presentations and discussions, and after addressing any issues, we decided to enforce Sorbet on our continuous integration system. Everyone was on board with the idea.
00:21:37.440 Stats generated by the Spoon tool reveal that only eight percent of our code files remain untyped, with seventy percent of our method calls invoking typed code. While we've made significant progress, there is still much work to do.
00:22:18.720 An example of dealing with dynamic code arises from our service objects, which encapsulate complex logic or actions with side effects. These are typically initialized with parameters, followed by a public call method.
00:23:04.000 To enhance code reusability, we included a helper to allow calling service objects via a class method. However, Sorbet struggles with this dynamic approach since it can't infer types from dynamic constructs.
00:24:02.239 To overcome this, we explored options for generating RBIs but quickly discovered that re-declaring types for every service object would be cumbersome. Thankfully, we found the library Parlor, which automates this process seamlessly.
00:24:51.680 After implementing this solution, we can now generate dynamic RBIs for all service objects, retaining clearer typing in our codebase. While we had to remove some dynamic imports and reconsider other libraries like Dry Monads, this shift has yielded better results for us.
00:25:50.000 Switching to Sorbet has allowed us to neatly replace constructs from Dry Monads with Sorbet features, ensuring our code remains comprehensible and maintainable.
00:26:17.920 While automation has improved our process, the current challenges include generating RBIs, maintaining dynamic imports, and automating various checks to ensure everything is consistent.
00:26:46.080 Runtime checks need further refinement, but we’re aiming to gradually implement them as we type more code. The journey with Sorbet has taught us that while gradual typing allows for flexibility, caution is required.
00:27:33.360 The inspection capabilities of Sorbet aid in understanding type returns and implementing better practices. Nevertheless, Rails' dynamic nature can still complicate interactions.
00:28:07.680 Keep in mind, many experienced Ruby developers may take time to adjust to a typing paradigm.
00:28:12.960 Lastly, it's important to note that Sorbet is still a work in progress, consistently refining its features and addressing evolving needs. We’ve faced various breaking changes with new versions, but we've remained committed to its use.
00:29:03.119 After two years of utilizing Sorbet, the team has recognized its value, despite any challenges faced. The parts of our code that now benefit from type checks have reduced the kinds of issues we're encountered before.
00:29:32.959 Finally, we're hiring! We've experienced significant growth in the past year, doubling in size and expecting that trend to continue, all while maintaining a great work-life balance and inclusive remote environment.
00:30:06.080 I want to share candidly that this has been the best employer of my career. If you're interested, please visit grailed.com/jobs or come talk to us at our booth.
00:30:33.920 For those seeking scholarly insight, feel free to browse our recommended research papers on static typing.
00:30:48.960 Thank you so much for being here today! If you have any questions, please come talk to me—I'll be here. Thank you!
00:30:55.680 So much.
Explore all talks recorded at RubyConf 2021
+95