Tapioca
Reflecting on Ruby Reflection for Rendering RBIs
Edit
#sorbet
#rbi
#dynamic-typing
#rubygems
#type-checking
#tapioca
Summarized using AI

Reflecting on Ruby Reflection for Rendering RBIs

Ufuk Kayserilioglu • September 04, 2020 • online

In the video "Reflecting on Ruby Reflection for Rendering RBIs," speaker Ufuk Kayserilioglu discusses the challenges and solutions involved in generating Ruby Interface files (RBIs) for gem dependencies at Shopify as part of their adoption process of Sorbet, a static type checker for Ruby.

Key Points:
- Introduction to Sorbet: Kayserilioglu explains that the adoption project began upon his joining Shopify in 2019, focusing on implementing gradual typing in their Rails monolith.
- Problem with RubyGems: Sorbet does not automatically analyze gem dependencies, which leaves it unaware of the constants and modules exported by gems, complicating the type-checking process.
- Example of Type-checking Challenge: To illustrate this issue, he presents an example involving a MyCLI class subclassing from Thor, where Sorbet couldn't recognize Thor or its methods.
- Role of Ruby Interface Files (RBIs): RBIs declare classes and methods without going into implementation details. This enables Sorbet to type-check user code so long as it knows the structure of the employed gems.
- Automation of RBI Generation: Given the number of dependencies (over 400 gems), Kayserilioglu emphasizes the necessity of automating the generation of RBI files using Ruby’s reflection capabilities.
- Understanding Reflection: He describes reflection as the ability to inspect and manipulate class properties and behaviors at runtime, which is essential for creating RBIs efficiently.
- Introspection Techniques: The discussion includes methods to programmatically gather information about classes, their methods, and relationships to dynamically build the necessary RBI files.
- Challenges of Dynamic Code: Kayserilioglu highlights potential pitfalls, such as changes in constants during runtime and complications arising from private constants that can lead to errors in understanding class hierarchies.
- Conclusion and Learning: He concludes that successfully generating RBIs while navigating Ruby's dynamic nature enhances development processes and teamwork. Furthermore, he encourages the audience to explore the open-sourced tapioca gem for more extensive insights. The talk finishes with an invitation for engagement on the topic.

Overall, the video emphasizes the innovative use of Ruby’s reflection capabilities to streamline the adoption of Sorbet and ensure thorough type-checking within the Shopify codebase.

Reflecting on Ruby Reflection for Rendering RBIs
Ufuk Kayserilioglu • September 04, 2020 • online

As part of our adoption process of Sorbet at Shopify, we needed an automated way to teach Sorbet about our ~400 gem dependencies. We decided to tackle this problem by generating interface files (RBI) for each gem via runtime reflection.

However, this turns out to not be as simple as it sounds; the flexible nature of Ruby allows gem authors to do many wild things that make this Hard. Come and hear about all the lessons that we learnt about runtime reflection in Ruby while building "tapioca".

RubyKaigi 2020 Takeout

00:00:01.199 Hi, welcome to RubyKaigi Takeout 2020.
00:00:04.240 I hope you're enjoying all the content.
00:00:06.399 Today, I will be reflecting on Ruby reflection for rendering RBIs.
00:00:14.080 My name is Ufuk Kayserilioglu, and I'm a Production Engineering Manager on the Ruby Infrastructure Team at Shopify.
00:00:21.119 A quick introduction to the concept: When I joined Shopify at the beginning of 2019, the first project I worked on was the adoption of Sorbet on our codebase.
00:00:27.680 For those of you who don't know, Sorbet is a static type checker built by the fine people at Stripe.
00:00:36.960 In the project I was involved with, we were trying to adopt gradual typing using Sorbet on our Rails monolith.
00:00:52.239 As we were working on this adoption, we quickly realized that we had a problem with RubyGems.
00:01:00.320 The issue was that Sorbet doesn't look into your gem dependencies, meaning it has no way of knowing what has been exported from those gems.
00:01:12.720 It only type checks your user code, which is problematic because the cost of entry for gradual typing adoption with Sorbet is for all references to constants in your code to be resolved.
00:01:21.280 If you refer to a constant or a module in your code, Sorbet needs to know what that constant is, but unfortunately, because it doesn't know the constants coming from gems, it has no visibility into them.
00:01:36.159 Let me provide a quick example to illustrate this.
00:01:47.840 For instance, if you have a MyCLI class that subclasses from Thor and calls the class option method, when you try to type check this with Sorbet, Sorbet will be confused because it won't recognize Thor or the class option method.
00:02:07.520 Without knowing the specifics of the class and its parameters, Sorbet can't successfully type check it because it cannot verify if 'assemble' is the correct type.
00:02:11.599 The way to solve this issue, developed by the Stripe team, was the concept of Ruby Interface files, or RBIs.
00:02:46.720 Ruby interface files are essentially Ruby files that declare classes and methods without all the implementation details.
00:02:59.760 This means that if you provide Sorbet with an RBI file that specifies what classes and methods exist, Sorbet can successfully type check user code that references those symbols.
00:03:19.200 However, creating and maintaining these RBI files manually for the over 400 gems we depend on at Shopify is a nightmare.
00:03:39.440 We needed a way to automate the generation of these files using Ruby’s reflection capabilities, which allows us to dynamically inspect and manipulate the structure of our Ruby classes and modules.
00:03:55.920 Reflection is the ability of a program to examine or manipulate the type properties, methods, or values of an object at runtime.
00:04:02.400 We often use reflection in Ruby, often unknowingly, as every time you call define_method or alias_method, you are engaging in reflection.
00:04:11.760 For our purpose, we focused on the introspection aspect, which allows us to read information about classes and methods without modifying them.
00:04:36.800 Before we start, it's important to understand that, in Ruby, everything is treated as a constant.
00:04:50.560 When declaring a class or module, you're actually creating a constant, and there’s a notable relationship between classes and constants.
00:05:04.960 For example, defining a class does not just create a class; it creates a constant that refers to that class.
00:05:22.000 This is critical because we will need to inspect these constants from our code to understand their relationships and generate the corresponding RBIs.
00:05:35.200 We’ll take a look at the methods and constants in the Store class as an example.
00:06:00.000 Using Ruby reflection, we can programmatically obtain the information required to construct our RBIs.
00:06:23.760 The key to this process is accessing constants by name, checking if they exist, and then retrieving references to them for further inspection.
00:06:46.480 Once we have references to the relevant classes, we can query them for their methods, superclass, constants, and included modules.
00:07:03.440 By retrieving this data, we can generate the corresponding structure for our RBIs, which is essential for enabling Sorbet to type-check the gem code our application depends on.
00:07:21.600 Each section of this process relies on Ruby's reflection mechanisms to access all necessary code elements dynamically.
00:07:45.440 One common obstacle we face in runtime reflection is dealing with dynamic code, where constants or methods may change during runtime.
00:08:12.480 We must ensure that our reflection code can handle such cases gracefully to avoid errors in our automated RBI generation.
00:08:36.720 As we continue, I will discuss the key aspects of using Ruby reflection effectively, while also addressing potential pitfalls that developers should be aware of.
00:08:54.880 One such pitfall is the misuse of constants, where classes redefine their own identity or functionality in ways that complicate our ability to introspect.
00:09:15.920 We must be vigilant against those who may try to create convoluted hierarchies or override critical methods like 'name' and 'superclass' without adhering to the expected norms.
00:09:45.760 Contending with these discrepancies can lead to errors in type-checking and hinder our development progress, as Sorbet will fail to correctly understand the constant’s behavior.
00:10:02.640 Another consideration is handling private constants, especially with classes where the superclass is private.
00:10:28.640 In this case, Sorbet may not properly assess the class hierarchy, leading to confusion and errors that could easily have been avoided.
00:10:52.720 We utilize reflection to check and bypass these methods when necessary, ensuring a smoother experience when generating RBIs.
00:11:09.920 Now, let's delve into how we can dynamically create and utilize a comprehensive set of RBIs for various gems, employing both manual insights and automated processes.
00:11:29.680 This interaction opens the door for a streamlined integration of third-party gems into our projects, allowing for effective use of type-checking throughout our codebase.
00:11:57.920 When working with dynamic languages such as Ruby, it's crucial to rely on both introspection and reflection to maintain flexibility and resilience against potential issues.
00:12:23.760 In summary, we can successfully generate Ruby Interface Files while mitigating the complexities introduced by Ruby's dynamic nature, thereby leading to a more robust development experience.
00:12:39.840 It's an evolving learning process that fosters strong collaboration and innovative resolution of challenges faced on a day-to-day basis.
00:13:01.440 To conclude, I encourage you to explore the options available for enhancing your development processes using Sorbet, and oversee your Ruby code with clarity and confidence.
00:13:31.520 If you're interested in leveraging the capabilities discussed today, I invite you to check out the tapioca gem which we have open-sourced for your convenience.
00:14:04.960 You can access it through GitHub and, of course, feel free to engage with me for inquiries or feedback related to this topic.
00:14:28.720 Thank you for your time. It was a pleasure speaking at RubyKaigi Takeout 2020. I hope to see you around!
Ufuk Kayserilioglu
@paracycle
Explore all talks recorded at RubyKaigi 2020 Takeout
+21