Keynote: Functional Future Ruby by Yukihiro Matsumoto

Talks

Yukihiro "Matz" Matsumoto

#functional-programming

#type-checking

#garbage-collection

Keynote: Functional Future Ruby by Yukihiro Matsumoto

by Yukihiro "Matz" Matsumoto

In his keynote address at Euruko 2019, Yukihiro Matsumoto, the creator of Ruby, discusses the future of the Ruby programming language, notably focusing on its functional enhancements. Matsumoto emphasizes the importance of open-source collaboration and community progress to keep Ruby relevant. With the planned release of Ruby 3.0 set for December 2020, he outlines several key components aimed at improving performance, static type checking, and concurrency models.

Key Points Discussed:

Open-Source Commitment: The importance of continuous community effort to advance Ruby's development is highlighted, noting that stagnation could lead to decline.
Static Analysis: While the Ruby community has debated the addition of type annotations, Matsumoto expresses his preference against them. Instead, Ruby 3.0 will introduce static type checking through separate type definition files, aiming to enhance the language without altering its core nature.
Type Profiler: A new component, the type profiler, will collect type information during application execution and help detect type conflicts, improving static analysis capabilities.
Performance Improvements: Matsumoto details the ongoing enhancements in Ruby's performance, stating that Ruby 2.6 is already significantly faster than its predecessors. He mentions improvements in garbage collection, the implementation of a Just-In-Time (JIT) compiler, and ongoing work towards object compaction in Ruby 3.0.
Concurrency Model Enhancements: Recognizing the need for better concurrency handling, Matsumoto discusses the introduction of lightweight threads and concepts from functional programming to improve Ruby's concurrency model, addressing challenges with multi-core processing.
Functional Programming Features: The keynote also explores potential new features inspired by functional programming, such as numbered block parameters and pattern matching to simplify coding tasks. Additionally, a pipeline operator is under consideration to enhance function composition and readability.
Continuous Evolution: Matsumoto concludes that the evolution of Ruby must continue to meet changing developer needs, ensuring the language remains relevant and useful.

The overall takeaway emphasizes a bright future for Ruby, driven by community engagement and innovative programming practices aimed at increasing efficiency and usability.

00:00:05.680 Matz is the creator of Ruby. He has spent 27 years of his life without Ruby and now spent 27 years with Ruby. For those last 27 years, he has helped promote developer happiness and has been a big proponent of the Ruby 3x3 performance gain. Today, he is going to talk to us about the functional future of Ruby. Take it away, Matz!

00:00:45.760 Let me first address a couple of things. First, I'm suffering from very bad jet lag. Secondly, I realized last night that my keynote is 30 minutes long; I thought it was 40 minutes. So, I'm kind of in a hurry. Our future will be open-source, and the community cannot stop because we have to keep moving forward or we will die. The future of Ruby lies in Ruby 3, which we are currently planning to release as version 3.0 in December next year, 2020. We are working on filling the missing pieces of the Ruby language, focusing on enhancing performance, addressing multi-core processing, and improving concurrency models.

00:02:09.440 First, let me talk about static analysis. In recent years, programming languages with static typing such as Rust, Go, and Swift have become very popular. As projects grow, tests become burdensome; their size increases and execution time lengthens. But tests are not inherently dry, meaning they often lead to redundancy. Human beings have not yet invented a way to create correct software without tests, so we need to find ways to reduce the burden of writing them. Other languages like PHP have moved toward type hinting, Python 3 has type annotations, and JavaScript has TypeScript. While some Ruby community members suggest adding type annotations, I disagree with this direction.

00:03:09.680 I dislike type annotations because they are not dry. However, Ruby 3 will include static type checking components. The first component will include type definition syntax, separate files for libraries, and type profilers. A person named Sato Matsumoto, who is not my relative, proposed static type checking in Ruby and abandoned the idea before. This is now his third attempt. The type definitions will describe types, including argument types, return value types, class modules, generic types, and interfaces. We will not change the Ruby language; rather, we will add type description files for libraries and standard libraries, as well as gems.

00:04:41.040 The next component is the type profiler, which is being developed by users such as Skandal. The type profiler is a key component for static analysis in Ruby 3. It uses abstract interpretation to virtually execute your application with type information. For example, if a method is called with an integer argument but another call tries to add a string, you would receive an error. The type profiler collects type information from your application and detects conflicts with existing type information from standard libraries and gems. If the type checker works well, it can generate the RBI files necessary for your application.

00:05:46.080 However, our type profiler is not yet very intelligent as it currently cannot handle generic types or distinguish between uniform and parameterized types. Your knowledge of your application will be required for refining generated RBI files. We may provide an optional file generator to use type information from your documentation, but this is still a plan and not yet in progress. We currently have two level-two static type checkers, including the original LDL. The first is Sorbet from Stripe, which has recently been open-sourced. Sorbet's current implementation requires some form of DSL to annotate your code.

00:07:00.400 In the future, Sorbet will be able to read RBI files, similar to those created by Sato Matsumoto. These static type checkers use different methodologies but have unique attributes. For example, Sorbet is fast and primarily written in C++. The type system for Sorbet largely relies on nominal types. We are collaborating with the Sorbet team to provide RBI support and are also working on another type checker called Steep, which utilizes structural typing and is written in Ruby, allowing for more flexibility and experimentation.

00:08:29.680 I expect a healthy competition between these two teams, as we explore static type analysis components that will enable Ruby programs to be type-checked statically without requiring additional type annotations in your Ruby code. Your existing Ruby programs will function as they are, without needing to add new type definitions. However, if you refine the type definition files, you'll acquire a more precise type-checking experience. Wouldn't that be great? I believe that adding type annotations to the Ruby language will not alter its core characteristics. I love Ruby just as it is!

00:09:39.280 Now, let’s discuss performance improvements. People often complain about the performance of various programming languages. Since Ruby 2 was released, we have been working to enhance performance consistently. In the past versions, Ruby's performance has improved significantly; Ruby 2.6 is nearly 1.8 times faster than its predecessors. However, we are constantly striving to enhance speed further. Traffic management is crucial in web applications, so we need to optimize memory, which is often the first bottleneck in web applications. To address this, we have improved garbage collection (GC) performance, including a generational garbage collector and incremental GC. We plan to introduce object compaction in version 3.0.

00:11:01.120 Additionally, we are addressing CPU bottlenecks with the Just-In-Time (JIT) compiler, introduced in Ruby 2.6. If you're using Ruby 2.6 or higher, you can activate the JIT compiler for CPU-intensive tasks. Benchmarks show that certain applications run up to 2.8 times faster with the JIT compiler. However, for common Rails applications, the CPU is usually not the main bottleneck. The memory model typically leads to performance issues. To tackle this, we are looking into lightweight compilation methods. Vladimir Mazalov, the original creator of the MJ compiler, is also working on a lightweight JIT compiler, which is highly anticipated.

00:12:03.040 Another important aspect of performance is the concurrency model. I regret to inform you that when I created Ruby in the 90s, I had no predictions about the rise of multi-core computers. How to effectively utilize threads has always been a challenge. Therefore, I believe we need to enhance our concurrency model to create a better abstraction for users. We are implementing a new concurrency model that will include concepts such as gills or isolates, which serve as lightweight threads. Unlike Elixir, we cannot declare all Ruby objects as immutable; instead, we will be adding new entities to enhance concurrency.

00:13:42.160 Improvements in concurrency will draw inspiration from functional programming languages, as they often embrace static typing, dynamic typing, and advanced concurrency models. Since my original lack of knowledge about these languages led to limitations, I’m now eager to involve functional programming benchmarks in enhancing Ruby's capabilities. Consequently, we are working on adding other features to be more compatible and competitive with our functional programming peers.

00:14:48.000 One prominent idea is introducing numbered block parameters, similar to Scala, Closure, and Groovy. This is an exciting opportunity for improvement, although I'm not entirely satisfied with every approach due to concerns about divergent design principles. As we approach the release of Ruby 2.7 in December, we have time to refine our ideas and possibly review other concepts. We can explore options like alternative syntax for parameter matching, which ties back to some foundational languages like Scheme.

00:17:18.560 We are also considering introducing pattern matching—not just through regular expressions, but functional programming pattern matching, which could significantly simplify JSON operations. By allowing for clearer syntax and better conditions in the patterns, we can make our language substantially more flexible going forward. Currently, this feature has already been merged into the Ruby 3 trunk, so you can anticipate future trials and experimentation with this.

00:19:09.600 An exciting feature we are exploring is a pipeline operator, inspired by functional programming paradigms like F#. We are aware that the implementation nuances can differ across languages, which presents unique challenges for us. For instance, function definitions and primary arguments' placements vary significantly between languages, necessitating adjustments in Ruby. The addition of the pipeline operator will also improve readability, allowing for clearer function flows in Ruby applications.

00:22:07.600 To facilitate this, there is an ongoing effort to allow comments and new lines in method chains, which is crucial for enhancing code clarity. This notion of breaking up method chains will let developers maintain readability while writing complex sequences. We are also introducing right-side assignments to add greater flexibility to Ruby programming. Despite challenges that Ruby faces, I firmly believe we must sustain its relevance in the ever-changing tech landscape. As a full-time Rubyist, my survival depends on it—if Ruby fades, so will my livelihood. Hence, our collective journey for survival continues as we work towards bettering the Ruby ecosystem.

00:24:42.760 I remain dedicated to ensuring the Ruby community thrives, making us happier and contributing to a better world. Thank you for being part of this journey. Thank you so much, Matz. That talk was fantastic! It's great to see the future of Ruby shining brightly, filling us with joy. To express our appreciation, here's a speaker gift from the team at Euruko. Thank you very much!

EuRuKo 2019