00:00:12.080
Hello everybody, welcome to RubyConf. This is a talk about Sorbet, the type checker for Ruby 3 that you can use today. I'm Dmitry Petrashko.
00:00:19.050
And I'm Jacob Zimmerman. Thanks for coming here. Here is the outline of this talk.
00:00:24.539
First of all, we'll give you some background on how this type checker was created, what problems it aims to solve, and why we believe it is the right solution.
00:00:30.990
We'll follow that with a discussion of the current reality of Sorbet, including how the community is using it today and how other companies are benefiting from it. Finally, we'll look at what is in store for us with Ruby 3.
00:00:41.760
First of all, what is Stripe? We're coming from Stripe, a company that helps others run their businesses online.
00:00:49.589
We help them focus on building their products by making it easier for them to facilitate payments on the internet. Stripe has more than 2,000 employees and engineering teams in San Francisco, Seattle, Dublin, and many other cities.
00:01:03.659
We are hiring interns and full-time engineers, as well as remote workers. Please visit stripe.com/jobs to see all the positions available.
00:01:09.780
Specifically, we are members of the Stripe Developer Productivity team, responsible for helping engineers have the most productive work life at Stripe.
00:01:22.440
We help them focus on building the product rather than overcoming obstacles specific to development.
00:01:29.130
More specifically, we use Ruby extensively, as it is our main product language.
00:01:35.880
As a Developer Productivity team, we assist Stripe engineers in being more productive by mostly working in Ruby and improving Ruby abstractions for Stripe.
00:01:41.820
Our customers are the internal Stripe engineers. There are hundreds of them writing millions of lines of code in a single repository.
00:01:46.950
This approach is intentional; we want to make it easier to implement changes by having a single repository with tens of millions of lines of code.
00:01:58.110
Thousands of changes happen in this repository every day by all those engineers.
00:02:04.620
To prioritize our work and figure out how to make our engineers more productive, one of the ways we learn about things worth improving is by asking them.
00:02:09.649
We run something called the Developer Productivity Survey. Historically, before Sorbet existed, a common piece of feedback we received was that engineers were taking a long time to get feedback from the existing system regarding whether their code was correct.
00:02:27.590
It took too long to comprehend unfamiliar code, like when reading code from other teams or from previous engineers. The codebase was generally fragile, and many unspoken assumptions contributed to this fragility.
00:02:39.890
As a result, slight changes could break something without notice. We needed to assess what options were available to improve this situation.
00:02:52.670
One option was to do nothing, ignore the feedback, and allow engineers to cope with the situation. The downside of this approach is that engineers would be less productive and would face the ongoing complexity of the codebase.
00:03:06.170
Another alternative was to treat the symptoms of the existing code by identifying the biggest offenders and manually fixing them. However, we believed that as the company grew, it would become increasingly difficult for a single team to manage this.
00:03:18.470
Declaring bankruptcy and rewriting everything was another option, but it posed a high risk as it would require hundreds of engineers to write the product while maintaining both the old and the new code.
00:03:26.180
We could also write targeted fixes with a few engineers, but agreeing on what needs fixing for the long term can be challenging. Often, priorities shift, and things that were once considered important may no longer be so.
00:03:35.880
This made it difficult to determine what to improve, and the engineering effort required was substantial.
00:03:42.080
Ultimately, we opted to create a type checker. The advantage of a type checker is that it's a tool, not a process that must be followed. It's beneficial because it is usable regardless of the existing process.
00:03:50.060
We started with a team of three engineers, which allowed for minimal investment while delivering incremental value month over month.
00:03:57.090
Improvements could be made on a file-by-file basis, and we saw daily advancements rather than needing a significant leap of faith.
00:04:05.050
This is why we developed the type checker.
00:04:11.380
In October 2017, we officially kicked off the project. We evaluated other alternatives, notably TypeRuby, a type checker for Ruby built by GitHub.
00:04:20.950
While we did not use it, we did take the parser from it, which was quite effective.
00:04:27.680
We also evaluated RDL, created by Jeff Foster, which focused on types in Ruby.
00:04:34.300
Unfortunately, it was too slow for our code base, sometimes taking literal hours to execute a single type check.
00:04:41.490
As a result, we decided to implement our own type checker.
00:04:48.520
In February 2018, we typed our first production code called Renewal World, taking a significant piece of our code base.
00:04:56.160
As developers of the tool, we verified its usefulness by taking real code, attempting to type it, and observing the error messages.
00:05:01.790
To be honest, the first error messages were excessive, flagging issues even when the code was correct. This discontent stemmed from enforcing a coding style that users might disagree with.
00:05:09.440
In other cases, it permitted things we did not want, leading us to refine our approach.
00:05:17.190
This was a pivotal moment for enhancing the Sorbet experience, as we aimed to balance allowing users to write code while simultaneously checking for unsafe practices.
00:05:23.960
In June 2018, we required Sorbet for all engineers at Stripe. This was a careful transition, ensuring that any pushback from engineers was acknowledged.
00:05:40.950
If someone felt that Sorbet wasn’t working for them, we were willing to reevaluate and make necessary adjustments.
00:05:45.890
Fortunately, the majority of people enjoyed using it. While some were indifferent, as we continued to develop features that mattered to them, they also came around.
00:05:55.410
In 2019, after extensive beta testing with more than 40 companies, Sorbet was open-sourced and is now available for everyone to use.
00:06:06.020
Let's dive deeper into the problems that Sorbet was designed to solve and how it addresses them. Jake?
00:06:12.620
Thank you, Dmitry. Now, I’ll explain the actual problems Sorbet aims to tackle within a real codebase.
00:06:19.500
Dmitry mentioned the Developer Productivity survey we run at Stripe. Over multiple iterations of this survey, three recurring complaints emerged: it takes too long to grasp unfamiliar code, it takes too long to receive feedback after changes, and it's too easy to accidentally break things.
00:06:33.170
To illustrate the first point, I pulled this code snippet almost verbatim from Stripe's codebase, where we have a method named find_card_similarity.
00:06:47.300
Consider being tasked with debugging and trying to understand this code. Right away, we encounter uncertainty: what does the merchants argument signify? Is it a string ID for a unique merchant or an object representing a merchant instance?
00:07:04.850
This uncertainty makes it challenging to know how to manipulate it in subsequent code.
00:07:10.800
Furthermore, what does it mean when the result of a fetch call might be null? Understanding what to do with that condition is essential.
00:07:16.740
Our goal is to reduce this confusion and provide clarity through the use of types.
00:07:24.460
The second major feedback was that it took too long to receive feedback after modifying code.
00:07:32.310
Stripe strongly values test-driven development, leading to a vast number of tests. Running all these tests sequentially would take days on a single machine.
00:07:41.050
To address this, we worked hard to parallelize these tests, utilizing a robust CI system that can yield results in 10 to 20 minutes.
00:07:54.230
While there have been significant improvements, that's still around five minutes of waiting to verify changes, a frustrating delay.
00:08:01.010
We aim to measure ourselves based on how quickly changes can be made and appraised.
00:08:08.590
Thirdly, it’s too easy for developers to unintentionally break things.
00:08:15.470
Continuing with the example from before, say we determine the argument should be a merchant instead of a string.
00:08:22.550
How many other places in the code need to be updated? Locating and updating every reference may not be straightforward.
00:08:31.480
While tests might catch some errors, they may not be exhaustive enough, leading to scenarios where seemingly simple changes lead to production issues.
00:08:38.870
These are the challenges we sought to solve with Sorbet.
00:08:46.830
Since deploying Sorbet, our engineers at Stripe have seen relief from these frustrations.
00:08:54.500
To demonstrate how Sorbet tackles these problems, we created an online sandbox for users to interact with.
00:09:01.350
This allows for hands-on experience with the features.
00:09:04.780
One crucial benefit is how Sorbet addresses the common concern of easing the understanding of unfamiliar code.
00:09:12.160
By simply hovering over the name of a method, you can instantly see its type and associated documentation.
00:09:20.620
This clears up ambiguities around whether a parameter is a string or an object.
00:09:26.970
With real-time feedback directly integrated into the editing experience, developers can quickly clarify the functionality and proper usage of unfamiliar methods.
00:09:37.440
If, upon changing code, an error emerges, Sorbet highlights the mistake immediately as you type.
00:09:45.450
This minimizes the need to wait for the results of lengthy test runs, allowing developers to receive real-time feedback.
00:09:53.040
Sorbet provides suggestions to fix issues as you code, enhancing productivity.
00:10:01.200
When working within large codebases like ours, encountering confusion is inevitable. However, Sorbet provides the tools needed to navigate and resolve these concerns swiftly.
00:10:08.081
Lastly, Sorbet helps ensure a level of safety as developers work. By embedded type signatures, you give future engineers a clear understanding of parameter expectations and desired outcomes.
00:10:14.300
This not only benefits the initial developer but streamlines the onboarding process for new team members.
00:10:21.880
Thus, to summarize, the three key benefits are: it makes code easier to understand, it provides immediate feedback, and it promotes better code hygiene through type annotations.
00:10:29.610
Now, I will hand it back to Dimitry, who will discuss the growth of the Sorbet community since our open-sourcing.
00:10:37.370
Thank you, Jake. As previously discussed, we opened Sorbet source code less than a year ago.
00:10:43.370
Even before that, there was an ongoing discussion with the Ruby core team, particularly with Matz, about integrating types into Ruby.
00:10:51.370
Ruby 3 will ship with type signatures in the standard library as a result of this collaboration.
00:10:57.370
This dynamic collaboration has included conversations involving creators of other type checkers such as Steep and RDL.
00:11:04.010
The biggest users of the type checker, other than Stripe, include Shopify, and they are actively participating in the discussions surrounding Ruby's adoption of types.
00:11:11.450
This partnership is instrumental in evolving Ruby's type system while ensuring it remains idiomatic.
00:11:18.860
Soutar Matsumoto, the creator of Steep, has significantly contributed to the syntax for type signatures that Ruby 3 will include.
00:11:25.700
Since we open-sourced, we’ve had more than 800 pull requests to the new repository for Sorbet.
00:11:34.870
Many of these contributions have come from outside of Stripe, with 135 contributors making an impact.
00:11:43.330
Among them, ten are part of the Sorbet team, thirty are Stripe engineers, and eighty-five are from various other companies.
00:11:50.350
This collaboration is helping ease the transition for users and discover additional errors that matter to them.
00:11:59.170
We believe that the tooling behind Sorbet crafted by these companies will assist you in easily adopting it and leveraging its benefits incrementally.
00:12:08.160
I will now turn the floor to others who will share more about the steps they've taken to benefit from Sorbet in their organizations.
00:12:15.930
Firstly, I want to highlight a few contributors.
00:12:21.280
Alex Naps has contributed parser support for Ruby 2.5, helping it work correctly, even in tricky cases.
00:12:27.160
This support has also been extended to Ruby 2.6, demonstrating substantial contributions from external parties.
00:12:34.860
Additionally, Vario has built a way to automatically add documentation from the standard library and gems to Sorbet signatures.
00:12:41.790
This means that when you hover over methods, the documentation for standard library functions is also readily available.
00:12:48.960
In response to feedback, CoinBase, as an early user of Sorbet, has built support for type definitions for gems.
00:12:55.800
Since open-sourcing Sorbet, 117 different gems have been added, making many popular libraries now type-safe.
00:13:02.300
This collaborative effort enhances the experience for users by providing auto-generated type signatures.
00:13:11.180
If you are a library author or user, we invite you to join in this exciting development.
00:13:18.450
Various promising projects are extending the ecosystem around Sorbet.
00:13:24.060
One of the most significant projects is Sorbet-Rails, initiated by the Chan Zuckerberg Initiative.
00:13:31.050
This extension optimizes Sorbet's synergy with Rails applications, a crucial feature since most Ruby projects utilize Rails.
00:13:37.520
We've also introduced Sorbet-Yard, which generates type signatures from yard annotations.
00:13:44.660
If your codebase already uses yard to describe a majority of arguments, you can now convert them to Sorbet types, which can be checked effectively.
00:13:54.840
Lastly, there is Sorbet-Plugins, which creates a plugin ecosystem for Sorbet.
00:14:01.560
This can teach Sorbet about internal DSLs and coding standards unique to your organization.
00:14:07.090
Thus, adding types can streamline your team’s work while keeping your code manageable.
00:14:13.940
In closing, we want to share direct feedback received from Sorbet users. Many stories underscore how Sorbet has made developers’ lives easier.
00:14:23.190
A common theme is that the iteration cycle can be overly painful within large codebases, making changes feel risky.
00:14:30.240
The ability for developers to refactor with fast feedback loops leads to cleaner, more maintainable codebases.
00:14:37.670
Gradual adoption within smaller projects led to enhanced code quality.
00:14:43.260
A feature often cited as a favorite is 'Go to Definition.' This transformed how developers interact with Ruby code.
00:14:51.080
Reduced friction in navigating code led to improved interest and confidence in experimentation.
00:14:57.300
Ultimately, enhanced productivity allows teams to build more features and adapt existing ones.
00:15:06.210
Now, the question stands: what is next for Sorbet? Our team’s main focus is investing in editor tools.
00:15:13.090
This includes autocomplete and 'Go to Definition' features, aiming for seamless reference management.
00:15:20.790
We also plan to automate refactorings, such as class and method renames.
00:15:26.630
Another priority is maintaining the speed of Sorbet as it integrates deeper into our more extensive codebase.
00:15:34.550
We've received valuable feedback from both internal and external users to those features that should receive our attention.
00:15:43.960
For instance, we have interfaces for modules and classes, final methods, final classes, exhaustive checks, and sealed classes.
00:15:51.320
There’s an evident demand for these features, and we are prioritizing their development.
00:16:00.050
Up next, we will have a presentation from the Shopify team detailing how they adopted Sorbet.
00:16:06.080
These discussions will help illuminate how to effectively use Sorbet across varying use cases.
00:16:13.090
We appreciate you attending, and if you haven't tried Sorbet yet, we encourage you to do so.
00:16:20.870
For those who have used it, please provide us with feedback on your experience.
00:16:27.740
We're eager to improve Sorbet with your insights. Thank you for coming. We’re hiring. I’ll be around, and don’t forget to grab stickers!
00:16:35.790
Are there any questions? I’m happy that we have plenty of time.
00:16:42.870
So the question was about the method signature annotation with sig that comes before the method.
00:16:48.640
Dmitry, would you like to take this one?
00:16:55.210
Sure! Our syntax has undergone several iterations. The concrete syntax 'sig' acts as a method name—this is hidden to reduce complexity.
00:17:03.480
There’s also a module at the bottom called T.sig which allows for extending any class.
00:17:10.700
Initially, we required all Stripe developers to incorporate this extension. However, it became clear that 80% of classes already had this extension.
00:17:18.320
We decided to streamline the process by integrating the capability directly into classes.
00:17:23.750
In essence, Sorbet is simply a Ruby library that leverages standard Ruby without special modifications.
00:17:29.190
The next question is how Sorbet integrates with Rails codebase hints.
00:17:35.700
Indeed, Sorbet implements the Language Server Protocol, making it compatible across multiple editors.
00:17:41.370
Thus it functions properly in both VI and Emacs.
00:17:48.050
For those using existing tools like RubyMine, the benefits of using Sorbet will become evident.
00:17:55.950
While RubyMine seeks to understand your codebase, Sorbet provides clear type definitions.
00:18:02.160
By articulating intent through type annotations, Sorbet offers guarantees instead of assumptions.
00:18:09.080
This guarantees a level of trust and security within your codebase.
00:18:15.400
In our operational context, the signatures from Sorbet are continuously verified at Stripe.
00:18:21.560
This approach ensures reliability and consideration in ensuring correctness.
00:18:28.393
There is ongoing discourse on whether to transition to structural types.
00:18:36.150
Though research indicates structural types hold potential, the user experience may suffer as they may present ambiguity between similar classes.
00:18:42.910
The focus remains on nominal types for their clarity and ease of use.
00:18:49.190
As we continue to examine the concept of structural types, side projects will evolve.
00:18:55.560
Gradual adoption has proven effective, leading us to recommend using untyped code areas.
00:19:02.360
Thank you for your insightful questions. We're delighted you were all here today.
00:19:08.790
As we look toward the future, we encourage everyone to share their feedback, explore Sorbet's capabilities, and help us improve further.
00:19:20.230
Given our limited time, we appreciate your engagement and look forwards to seeing the outcomes of Sorbet as it progresses.
