A Static Type Analyzer of Untyped Ruby Code for Ruby 3

Summarized using AI

A Static Type Analyzer of Untyped Ruby Code for Ruby 3

Yusuke Endoh • November 19, 2019 • Nashville, TN • Talk

The video titled "A Static Type Analyzer of Untyped Ruby Code for Ruby 3" by Yusuke Endoh, presented at RubyConf 2019, explores the development of a static type analysis tool for Ruby. This presentation emphasizes improving the Ruby programming language by introducing static typing features and tools like the Type Profiler.

Key Points Discussed:
- Speaker Background: Yusuke Endoh is a long-time contributor to Ruby, known for implementing keyword arguments and creating benchmarks for Ruby 3.
- Introduction to Ruby 3 Plans: The Ruby team aims to enhance the language with a comprehensive static type checker framework to support developers in using type analysis tools effectively.
- Type Signatures: A unified standard type signature format is proposed to improve interoperability among various type checkers and to provide clarity in Ruby code.
- Type Profiler: This is the central tool discussed. Type Profiler analyzes unannotated Ruby code to infer type signatures automatically, helping developers avoid manual annotations. It works by abstractly running code to locate potential errors and suggest signatures without requiring explicit type definitions from the user.
- Goals and Challenges: The presentation outlines three main goals for the Type Profiler project:
1. Establishing a Ruby signature language as the standard format for type signatures.
2. Developing a feature to automatically generate signature prototypes.
3. Implementing a type checker to verify the consistency of code against the generated type signatures.
- RBS Introduction: Sotaro, a member of the Ruby team, joins in to explain RBS (Ruby Signature), a new language for defining the types of Ruby programs, which keeps type annotations separate to maintain clarity. This facilitates both type checking and better documentation.
- Types of Analysis and Results: By analyzing methods with practical examples, the Type Profiler aims to uncover type-related issues such as incorrect method parameters. The performance and efficiency of Ruby can also be enhanced through insights gained from the analysis, albeit some limitations remain regarding complex or unknown types.
- Real-World Applications: The need for Type Profiler to engage with actual Ruby codebases to validate inferred types and improve its efficacy is emphasized.

Conclusion: The presentation concludes with a strong encouragement for adopting Type Profiler as an invaluable tool for Ruby developers seeking to implement and understand static typing without the burden of manual annotations. Yusuke Endoh encourages ongoing exploration of static typing within the Ruby community, inviting questions and further engagement during the session.

A Static Type Analyzer of Untyped Ruby Code for Ruby 3
Yusuke Endoh • November 19, 2019 • Nashville, TN • Talk

RubyConf 2019 - A Static Type Analyzer of Untyped Ruby Code for Ruby 3 by Yusuke Endoh

Ruby 3 is planned to provide a toolchain for static analysis: (1) the standard type signature format for Ruby code, (2) a type analyzer to guess a signature of a non-signatured Ruby code, and (3) a type checker (such as Sorbet) to verify a code with its signature. In the talk, we present a candidate of (2), called Type Profiler, which aims to allow you not to write a signature manually. It abstractly runs a non-annotated Ruby code in "type" level, tries to find possible errors, and generates a type signature prototype for the code. We show some demos and development progress.

#confreaks #rubyconf2019

RubyConf 2019

00:00:12.290 So let's start. I took about yet another type analyzer.
00:00:19.619 This is the second time for me to talk at RubyConf.
00:00:29.449 Oops, okay, thank you. But my previous talk was about object-oriented programming.
00:00:38.249 So this is the first time for me to talk about static types.
00:00:44.370 Okay, so I'm a bit nervous, but I'll do my best.
00:00:50.210 I'm Yusuke Endoh. I became a contributor to Ruby in 2008.
00:00:57.510 I have contributed to Ruby, and one of the most famous features I implemented is keyword arguments.
00:01:03.629 I implemented keyword arguments in Ruby 2.0.
00:01:09.960 I've created a de facto standard benchmark program for Ruby 3 called 'doped carrot,' which has driven many early implementers.
00:01:16.380 I was involved in the design management and have been actively part of it until today.
00:01:25.320 Additionally, I work in Quality Assurance for Ruby, including testing.
00:01:32.070 As for my job, I'm developing Ruby with Cookpad.
00:01:38.250 You may know Cookpad as a robust template to create and share cooking content.
00:01:46.470 Let me briefly introduce our company: Cookpad is dedicated to making everyday cooking fun.
00:01:54.210 Our main service is a collaborative cooking platform where users can submit their original recipes.
00:02:00.240 You can enjoy this space that is used by approximately 90 million monthly unique users globally across 30 languages.
00:02:09.209 We currently have users in 73 countries, with the service continually expanding.
00:02:15.990 Our aim is to be number one in cooking services across 100 countries.
00:02:22.470 Cookpad is powered by Ruby, and to maintain this service, we need many excellent Ruby engineers.
00:02:30.630 Currently, I'm located in Japan, but our headquarters are in the United Kingdom.
00:02:38.640 So, if you're interested in joining our team, feel free to contact me.
00:02:47.990 Okay, today I'll talk about three main topics: Ruby 3's plans, type signatures, and type analyze tools.
00:02:55.500 As mentioned in the previous two talks in this room, we discussed static type checkers for Ruby and its evolution.
00:03:05.000 The goal is to create a comprehensive type checker framework that allows developers to choose the appropriate type analysis tool for their needs.
00:03:13.360 This is a key feature of integrating multiple checkers and defining a Ruby signature.
00:03:22.720 This standard format will help write type information for Ruby code that all type checkers can share.
00:03:30.030 As for our standard libraries and gems, I would like to talk about my project called Type Profiler.
00:03:37.440 Type Profiler is a type analyzer that accepts unannotated Ruby code and attempts to provide type inferences.
00:03:45.780 It helps you to develop and suggest a signature prototype.
00:03:53.250 Now, let me explain the goals of the profiler.
00:03:58.680 Its aim is to find possible values in an abstract running environment, improving the development experience.
00:04:12.660 The results mostly focus on providing insights for human consumption, so a few false positives might be acceptable.
00:04:19.290 For example, consider a method that is defined to accept one integer argument.
00:04:28.090 If it's mistakenly defined to accept a string instead, the goal would be to catch such errors before runtime.
00:04:39.890 The most traditional way to ensure type safety is to write explicit type annotations.
00:04:49.710 You would need to specify what types a method accepts and returns.
00:04:56.280 However, writing these annotations may not be desirable for everyone.
00:05:03.680 So, how many of you would prefer to write type annotations for your Ruby code? Please raise your hand.
00:05:13.500 Okay, thank you. And now, for those who do not want to write such annotations, please raise your hand.
00:05:20.360 Honestly, I had prepared two slides, expecting you would be inclined toward writing type annotations.
00:05:31.870 Annotations are useful not only for type checkers but also as communication tools to clarify API usage.
00:05:39.350 This is especially significant in large-scale projects.
00:05:46.550 In Ruby 3, you can write types manually because it provides an official way to do so.
00:05:53.240 If you include type annotations, you gain the benefits of a strong type-checking system.
00:06:00.620 However, as Matt mentioned in his keynote, he still finds type annotations unappealing.
00:06:08.650 This is why I am working on Type Profiler, which allows you to avoid writing annotations by hand.
00:06:19.920 With this tool, you can check your code almost without limitations.
00:06:26.430 Unfortunately, developing Type Profiler is quite challenging.
00:06:33.060 I admit that it is still very preliminary.
00:06:40.380 Currently, the development of Type Profiler is facing difficulties.
00:06:49.990 To achieve our goals, we plan to provide three main features.
00:06:56.990 First, Ruby signature language will serve as the standard type signature format.
00:07:04.600 Second, we will implement a typing feature to generate signature prototypes.
00:07:12.570 Finally, there will be a type checker to verify the consistency between code and signatures.
00:07:20.620 Using the Ruby signature language, you can write type signatures for your Ruby code.
00:07:28.290 With the second item, you can automatically generate signature prototypes for your Ruby code.
00:07:35.230 Finally, through the third item, you can type-check your code.
00:07:42.360 So, in this talk, I will focus on explaining items one and two.
00:07:49.570 Now, let’s get to Ruby signatures. I asked Sotaro, the author of the signature language, to explain.
00:07:57.540 I will pass the mic to him. Oh, thank you.
00:08:04.590 I'm Sotaro, and I lead the design and implementation of RBS in the Ruby team.
00:08:10.310 I also developed my static type checker called Steep.
00:08:17.710 This year, we don't have such a thing as Steep, but I'm working for the Ruby core.
00:08:23.500 Now, let’s talk about RBS.
00:08:29.570 RBS is a standard language designed to describe the types of Ruby programs, including modules and class definitions.
00:08:36.990 The most important aspect of RBS is that it is a different language from Ruby.
00:08:43.430 It has a distinct syntax and semantic structures.
00:08:50.190 The key purpose is to maintain clean separation between type annotations.
00:08:58.400 Here’s an example of an RBS definition.
00:09:03.860 Within the RBS file, you can define a class named 'Increment' with a method to receive an integer and return an integer.
00:09:10.670 The syntax will look similar to Ruby—but it’s entirely different.
00:09:17.990 The type-related information is isolated in a separate file, unlike being mixed with the Ruby code.
00:09:26.220 RBS supports generics, overloading, optional types, and type blocks.
00:09:34.020 It also supports mixins, including modules and extensions.
00:09:41.780 RBS introduces the concept of interfaces.
00:09:49.570 The difference between modules and interfaces is that interfaces are not tied to specific Ruby classes.
00:09:56.220 Instead, they support a form of duck typing, implying that any type meeting certain method requirements could be used.
00:10:03.920 RBS serves two primary functions: type checking and documentation.
00:10:10.800 If you wish to type-check your application or library with static type checkers including Steep or Sorbet, RBS will describe the type information.
00:10:19.100 Additionally, RBS provides clearer documentation of classes and methods, making it easier to understand.
00:10:26.850 To create RBS files, you can either write them manually or use tools that generate them from existing Ruby code.
00:10:36.420 For instance, we provide tools to generate templates from Ruby code.
00:10:43.500 You can also generate RBS files using Type Profiler, the subject of this session.
00:10:50.300 Now I'm working on a tool to validate RBS definitions against Ruby programs dynamically.
00:10:56.520 This tool uses runtime API checks for type correctness.
00:11:01.420 This approach allows developers to write RBS without attempting exhaustive Ruby code compatibility.
00:11:07.540 Now, let's return to Type Profiler and explore its key concepts and demos.
00:11:17.740 The primary idea of this analysis is to utilize type inference in Ruby.
00:11:23.410 However, it comes with known limitations.
00:11:28.910 Type Profiler applies abstract interpretation to the code, generating potential type signatures from untyped Ruby.
00:11:43.610 For instance, it can analyze logic like the 'increment' method from earlier.
00:11:52.020 In case it works perfectly, the type inference would yield correct results.
00:11:57.590 Nonetheless, if type inference incorrectly interprets input, further investigation is required.
00:12:05.160 The primary focus is ensuring clarity and correctness, thus reducing the need for constant manual typing.
00:12:12.960 To illustrate, I ran two simple test cases to show how Type Profiler can operate.
00:12:21.750 The first test is a basic Ruby program of around 300 lines.
00:12:29.520 Applying Type Profiler to this code outputs the inferred signatures.
00:12:36.360 For example, a method to represent a 3D vector might indicate types such as integers.
00:12:43.730 The analysis may detect parameter types and return types associated with specific methods.
00:12:49.610 However, incorrect assumptions during inference may lead to inaccurate results.
00:12:55.570 Before applying analysis results, the user should check for any discrepancies.
00:13:02.400 Another important feature is the ability to catch bad type pairs.
00:13:11.570 Types would indicate errors like attempting to add an integer and a string.
00:13:19.380 Overall, Type Profiler aims to provide better understanding while navigating Ruby's type structure.
00:13:28.760 Nevertheless, some limitations remain, especially with unknown types or constructs.
00:13:35.450 Before explaining how Type Profiler analyzes Ruby code, let’s explore its capabilities.
00:13:44.520 Next, I will conduct a brief analysis of a complex program.
00:13:52.680 Analyzing the test will provide insight into Ruby's code structure.
00:13:59.840 The first program, Obscura, serves as the benchmark for Ruby 3 and is quite complex.
00:14:07.800 The second is a toy example showcasing fundamental Ruby features.
00:14:14.090 As we review these cases, we can see how Type Profiler effectively handles different coding aspects.
00:14:21.540 For example, the analysis of mathematical operations offers unique challenges.
00:14:30.700 By analyzing type signatures for methods, we learn about method definitions.
00:14:37.900 Such analysis showcases potential performance improvements through method specialization.
00:14:47.400 After this analysis, we identify opportunities to optimize the internals of Ruby.
00:14:54.590 However, many of these features still require additional context from existing Ruby constructs.
00:15:02.020 Despite generating useful insights, Type Profiler may misinterpret unknown classes and types.
00:15:09.820 Thus, it’s essential to regularly update the knowledge base behind Type Profiler.
00:15:15.300 Moving beyond basic examples, Type Profiler needs engagement with real-world applications.
00:15:22.650 This is imperative to validate types against complex Ruby architectures.
00:15:29.420 In conclusion, I appreciate your attention and interest in exploring static typing in Ruby.
00:15:36.590 If any questions persist, please feel free to reach out.
00:15:42.780 Thank you, and I hope to see you at the next RubyConf!
Explore all talks recorded at RubyConf 2019
+88