Static Analysis
A Static Type Analyzer of Untyped Ruby Code for Ruby 3
A Static Type Analyzer of Untyped Ruby Code for Ruby 3
by Yusuke EndohThe video titled "A Static Type Analyzer of Untyped Ruby Code for Ruby 3" by Yusuke Endoh, presented at RubyConf 2019, explores the development of a static type analysis tool for Ruby. This presentation emphasizes improving the Ruby programming language by introducing static typing features and tools like the Type Profiler.
Key Points Discussed:
- Speaker Background: Yusuke Endoh is a long-time contributor to Ruby, known for implementing keyword arguments and creating benchmarks for Ruby 3.
- Introduction to Ruby 3 Plans: The Ruby team aims to enhance the language with a comprehensive static type checker framework to support developers in using type analysis tools effectively.
- Type Signatures: A unified standard type signature format is proposed to improve interoperability among various type checkers and to provide clarity in Ruby code.
- Type Profiler: This is the central tool discussed. Type Profiler analyzes unannotated Ruby code to infer type signatures automatically, helping developers avoid manual annotations. It works by abstractly running code to locate potential errors and suggest signatures without requiring explicit type definitions from the user.
- Goals and Challenges: The presentation outlines three main goals for the Type Profiler project:
1. Establishing a Ruby signature language as the standard format for type signatures.
2. Developing a feature to automatically generate signature prototypes.
3. Implementing a type checker to verify the consistency of code against the generated type signatures.
- RBS Introduction: Sotaro, a member of the Ruby team, joins in to explain RBS (Ruby Signature), a new language for defining the types of Ruby programs, which keeps type annotations separate to maintain clarity. This facilitates both type checking and better documentation.
- Types of Analysis and Results: By analyzing methods with practical examples, the Type Profiler aims to uncover type-related issues such as incorrect method parameters. The performance and efficiency of Ruby can also be enhanced through insights gained from the analysis, albeit some limitations remain regarding complex or unknown types.
- Real-World Applications: The need for Type Profiler to engage with actual Ruby codebases to validate inferred types and improve its efficacy is emphasized.
Conclusion: The presentation concludes with a strong encouragement for adopting Type Profiler as an invaluable tool for Ruby developers seeking to implement and understand static typing without the burden of manual annotations. Yusuke Endoh encourages ongoing exploration of static typing within the Ruby community, inviting questions and further engagement during the session.
00:00:12.290
So let's start. I took about yet another type analyzer.
00:00:19.619
This is the second time for me to talk at RubyConf.
00:00:29.449
Oops, okay, thank you. But my previous talk was about object-oriented programming.
00:00:38.249
So this is the first time for me to talk about static types.
00:00:44.370
Okay, so I'm a bit nervous, but I'll do my best.
00:00:50.210
I'm Yusuke Endoh. I became a contributor to Ruby in 2008.
00:00:57.510
I have contributed to Ruby, and one of the most famous features I implemented is keyword arguments.
00:01:03.629
I implemented keyword arguments in Ruby 2.0.
00:01:09.960
I've created a de facto standard benchmark program for Ruby 3 called 'doped carrot,' which has driven many early implementers.
00:01:16.380
I was involved in the design management and have been actively part of it until today.
00:01:25.320
Additionally, I work in Quality Assurance for Ruby, including testing.
00:01:32.070
As for my job, I'm developing Ruby with Cookpad.
00:01:38.250
You may know Cookpad as a robust template to create and share cooking content.
00:01:46.470
Let me briefly introduce our company: Cookpad is dedicated to making everyday cooking fun.
00:01:54.210
Our main service is a collaborative cooking platform where users can submit their original recipes.
00:02:00.240
You can enjoy this space that is used by approximately 90 million monthly unique users globally across 30 languages.
00:02:09.209
We currently have users in 73 countries, with the service continually expanding.
00:02:15.990
Our aim is to be number one in cooking services across 100 countries.
00:02:22.470
Cookpad is powered by Ruby, and to maintain this service, we need many excellent Ruby engineers.
00:02:30.630
Currently, I'm located in Japan, but our headquarters are in the United Kingdom.
00:02:38.640
So, if you're interested in joining our team, feel free to contact me.
00:02:47.990
Okay, today I'll talk about three main topics: Ruby 3's plans, type signatures, and type analyze tools.
00:02:55.500
As mentioned in the previous two talks in this room, we discussed static type checkers for Ruby and its evolution.
00:03:05.000
The goal is to create a comprehensive type checker framework that allows developers to choose the appropriate type analysis tool for their needs.
00:03:13.360
This is a key feature of integrating multiple checkers and defining a Ruby signature.
00:03:22.720
This standard format will help write type information for Ruby code that all type checkers can share.
00:03:30.030
As for our standard libraries and gems, I would like to talk about my project called Type Profiler.
00:03:37.440
Type Profiler is a type analyzer that accepts unannotated Ruby code and attempts to provide type inferences.
00:03:45.780
It helps you to develop and suggest a signature prototype.
00:03:53.250
Now, let me explain the goals of the profiler.
00:03:58.680
Its aim is to find possible values in an abstract running environment, improving the development experience.
00:04:12.660
The results mostly focus on providing insights for human consumption, so a few false positives might be acceptable.
00:04:19.290
For example, consider a method that is defined to accept one integer argument.
00:04:28.090
If it's mistakenly defined to accept a string instead, the goal would be to catch such errors before runtime.
00:04:39.890
The most traditional way to ensure type safety is to write explicit type annotations.
00:04:49.710
You would need to specify what types a method accepts and returns.
00:04:56.280
However, writing these annotations may not be desirable for everyone.
00:05:03.680
So, how many of you would prefer to write type annotations for your Ruby code? Please raise your hand.
00:05:13.500
Okay, thank you. And now, for those who do not want to write such annotations, please raise your hand.
00:05:20.360
Honestly, I had prepared two slides, expecting you would be inclined toward writing type annotations.
00:05:31.870
Annotations are useful not only for type checkers but also as communication tools to clarify API usage.
00:05:39.350
This is especially significant in large-scale projects.
00:05:46.550
In Ruby 3, you can write types manually because it provides an official way to do so.
00:05:53.240
If you include type annotations, you gain the benefits of a strong type-checking system.
00:06:00.620
However, as Matt mentioned in his keynote, he still finds type annotations unappealing.
00:06:08.650
This is why I am working on Type Profiler, which allows you to avoid writing annotations by hand.
00:06:19.920
With this tool, you can check your code almost without limitations.
00:06:26.430
Unfortunately, developing Type Profiler is quite challenging.
00:06:33.060
I admit that it is still very preliminary.
00:06:40.380
Currently, the development of Type Profiler is facing difficulties.
00:06:49.990
To achieve our goals, we plan to provide three main features.
00:06:56.990
First, Ruby signature language will serve as the standard type signature format.
00:07:04.600
Second, we will implement a typing feature to generate signature prototypes.
00:07:12.570
Finally, there will be a type checker to verify the consistency between code and signatures.
00:07:20.620
Using the Ruby signature language, you can write type signatures for your Ruby code.
00:07:28.290
With the second item, you can automatically generate signature prototypes for your Ruby code.
00:07:35.230
Finally, through the third item, you can type-check your code.
00:07:42.360
So, in this talk, I will focus on explaining items one and two.
00:07:49.570
Now, let’s get to Ruby signatures. I asked Sotaro, the author of the signature language, to explain.
00:07:57.540
I will pass the mic to him. Oh, thank you.
00:08:04.590
I'm Sotaro, and I lead the design and implementation of RBS in the Ruby team.
00:08:10.310
I also developed my static type checker called Steep.
00:08:17.710
This year, we don't have such a thing as Steep, but I'm working for the Ruby core.
00:08:23.500
Now, let’s talk about RBS.
00:08:29.570
RBS is a standard language designed to describe the types of Ruby programs, including modules and class definitions.
00:08:36.990
The most important aspect of RBS is that it is a different language from Ruby.
00:08:43.430
It has a distinct syntax and semantic structures.
00:08:50.190
The key purpose is to maintain clean separation between type annotations.
00:08:58.400
Here’s an example of an RBS definition.
00:09:03.860
Within the RBS file, you can define a class named 'Increment' with a method to receive an integer and return an integer.
00:09:10.670
The syntax will look similar to Ruby—but it’s entirely different.
00:09:17.990
The type-related information is isolated in a separate file, unlike being mixed with the Ruby code.
00:09:26.220
RBS supports generics, overloading, optional types, and type blocks.
00:09:34.020
It also supports mixins, including modules and extensions.
00:09:41.780
RBS introduces the concept of interfaces.
00:09:49.570
The difference between modules and interfaces is that interfaces are not tied to specific Ruby classes.
00:09:56.220
Instead, they support a form of duck typing, implying that any type meeting certain method requirements could be used.
00:10:03.920
RBS serves two primary functions: type checking and documentation.
00:10:10.800
If you wish to type-check your application or library with static type checkers including Steep or Sorbet, RBS will describe the type information.
00:10:19.100
Additionally, RBS provides clearer documentation of classes and methods, making it easier to understand.
00:10:26.850
To create RBS files, you can either write them manually or use tools that generate them from existing Ruby code.
00:10:36.420
For instance, we provide tools to generate templates from Ruby code.
00:10:43.500
You can also generate RBS files using Type Profiler, the subject of this session.
00:10:50.300
Now I'm working on a tool to validate RBS definitions against Ruby programs dynamically.
00:10:56.520
This tool uses runtime API checks for type correctness.
00:11:01.420
This approach allows developers to write RBS without attempting exhaustive Ruby code compatibility.
00:11:07.540
Now, let's return to Type Profiler and explore its key concepts and demos.
00:11:17.740
The primary idea of this analysis is to utilize type inference in Ruby.
00:11:23.410
However, it comes with known limitations.
00:11:28.910
Type Profiler applies abstract interpretation to the code, generating potential type signatures from untyped Ruby.
00:11:43.610
For instance, it can analyze logic like the 'increment' method from earlier.
00:11:52.020
In case it works perfectly, the type inference would yield correct results.
00:11:57.590
Nonetheless, if type inference incorrectly interprets input, further investigation is required.
00:12:05.160
The primary focus is ensuring clarity and correctness, thus reducing the need for constant manual typing.
00:12:12.960
To illustrate, I ran two simple test cases to show how Type Profiler can operate.
00:12:21.750
The first test is a basic Ruby program of around 300 lines.
00:12:29.520
Applying Type Profiler to this code outputs the inferred signatures.
00:12:36.360
For example, a method to represent a 3D vector might indicate types such as integers.
00:12:43.730
The analysis may detect parameter types and return types associated with specific methods.
00:12:49.610
However, incorrect assumptions during inference may lead to inaccurate results.
00:12:55.570
Before applying analysis results, the user should check for any discrepancies.
00:13:02.400
Another important feature is the ability to catch bad type pairs.
00:13:11.570
Types would indicate errors like attempting to add an integer and a string.
00:13:19.380
Overall, Type Profiler aims to provide better understanding while navigating Ruby's type structure.
00:13:28.760
Nevertheless, some limitations remain, especially with unknown types or constructs.
00:13:35.450
Before explaining how Type Profiler analyzes Ruby code, let’s explore its capabilities.
00:13:44.520
Next, I will conduct a brief analysis of a complex program.
00:13:52.680
Analyzing the test will provide insight into Ruby's code structure.
00:13:59.840
The first program, Obscura, serves as the benchmark for Ruby 3 and is quite complex.
00:14:07.800
The second is a toy example showcasing fundamental Ruby features.
00:14:14.090
As we review these cases, we can see how Type Profiler effectively handles different coding aspects.
00:14:21.540
For example, the analysis of mathematical operations offers unique challenges.
00:14:30.700
By analyzing type signatures for methods, we learn about method definitions.
00:14:37.900
Such analysis showcases potential performance improvements through method specialization.
00:14:47.400
After this analysis, we identify opportunities to optimize the internals of Ruby.
00:14:54.590
However, many of these features still require additional context from existing Ruby constructs.
00:15:02.020
Despite generating useful insights, Type Profiler may misinterpret unknown classes and types.
00:15:09.820
Thus, it’s essential to regularly update the knowledge base behind Type Profiler.
00:15:15.300
Moving beyond basic examples, Type Profiler needs engagement with real-world applications.
00:15:22.650
This is imperative to validate types against complex Ruby architectures.
00:15:29.420
In conclusion, I appreciate your attention and interest in exploring static typing in Ruby.
00:15:36.590
If any questions persist, please feel free to reach out.
00:15:42.780
Thank you, and I hope to see you at the next RubyConf!
