Fibers

Summarized using AI

Ruby 3.0 and Beyond

Yukihiro "Matz" Matsumoto • April 07, 2021 • online

In the keynote speech titled "Ruby 3.0 and Beyond," delivered at the Ruby Day Conference 2021, Yukihiro "Matz" Matsumoto discusses the advancements brought by the release of Ruby 3.0, emphasizing enhancements in concurrency, performance, and productivity. The video explores the challenges of maintaining compatibility while introducing new features, as Ruby strives to balance progress and user needs.

Key Points Discussed:

- Release and Goals: Ruby 3.0 was released on December 25, 2020, aiming to improve concurrency, performance, and productivity.

- Compatibility: Nearly all Ruby programs will run on Ruby 3.0 without modification. Compatibility is crucial, as breaking it can fragment the community, as seen in other languages like Python and PHP.

- Performance Enhancements: Ruby 3 aims for "Ruby 3x3," meaning it should be three times faster than Ruby 2.x under certain benchmarks. Improvements include:

- Enhanced garbage collection strategies.

- Introduction of the Method Just-in-Time (MJIT) compiler, leading to significant speed enhancements.

- Real-world performance improvements were demonstrated through benchmarks, notably using the "Optcarrot" suite, showcasing a workflow increase from 32.47 to 98.36 frames per second with MJIT.

- Concurrency: Matz introduces Ractor and Async Fiber for true concurrency handling in Ruby, catering to I/O-bound and CPU-bound tasks respectively.

- Async Fiber allows for non-blocking operations, improving I/O performance.

- Ractor promotes isolation between concurrent tasks, which prevents data races, an advancement over Ruby's previous threading model constrained by a Global Interpreter Lock (GIL).

- Static Analysis and Typing: The introduction of RBS (Ruby Signature) aims to enhance error detection and support developers in writing safer code. TypeProf helps generate RBS data, assisting in better code clarity.

- Syntax Improvements: New syntax features include pattern matching, which enhances code expressiveness and readability.

- Community and Tooling: Matz emphasizes the importance of community contributions and the need for better tooling to support Ruby's evolution.

In conclusion, Matz envisions an ongoing evolution of Ruby 3.0 that embraces new features while maintaining a rich compatibility with previous versions. He urges developers to contribute to the community and collaborate for a better programming experience in Ruby.

Ruby 3.0 and Beyond
Yukihiro "Matz" Matsumoto • April 07, 2021 • online

Ruby3.0 and Beyond - Yukihiro "Matz" Matsumoto - rubyday 2021

The ninth edition of the Italian Ruby conference, for the third time organised by GrUSP, took place online on April 7th 2021.

Speaker and details on https://2021.rubyday.it/

rubyday 2021

00:00:00.120 Oh my gosh.
00:00:33.899 Foreign.
00:01:02.000 Thank you for having me at the Ruby Day Conference.
00:01:07.020 Due to the pandemic, it is very difficult to visit and see you face to face, but thanks to technology, we can communicate via the Internet.
00:01:13.500 Today, as a keynote speaker, I am going to talk about "Ruby 3.0 and Beyond."
00:01:20.040 Ruby 3.0 was released on December 25th, 2020, so last Christmas we received this big update.
00:01:26.100 The goal of this release was to improve concurrency, performance, and productivity.
00:01:33.240 Concurrency is accomplished by Ractor and Async Fiber, while performance improvements come from these enhancements in the compiler.
00:01:42.900 Productivity is targeted through Ruby signature RBS and the type graph.
00:01:50.579 The basic design goals of Ruby 3 are compatibility, speed, and overall better performance.
00:01:58.200 Let me explain them one by one.
00:02:09.979 Regarding compatibility, nearly all Ruby programs will run on Ruby 3.0 without any modification.
00:02:16.160 This is very important since compatibility matters.
00:02:21.300 As a language designer, I feel this every day. Designing a language is difficult.
00:02:28.440 Ruby should maintain a compatibility model.
00:02:34.220 However, at the same time, we need to advance, as we are an open-source community.
00:02:41.220 Our community members are not obligated to use Ruby.
00:02:47.340 They use Ruby because they love it, but once they become bored, they might switch to another programming language or community.
00:02:55.560 If we stop progressing, we risk dying as a community. That is the fate of open-source software.
00:03:03.300 It's a contradiction because we must keep compatibility, and yet we have to move forward.
00:03:08.459 As language designers, including myself, we make mistakes.
00:03:14.099 Yes, I wanted to fix past failures and mistakes.
00:03:20.940 However, sometimes the community relies on our design errors.
00:03:29.640 Fixing them could lead to chaos.
00:03:36.360 Once in a while, we release major versions like 2.0 or 3.0, which give us chances to fix past mistakes.
00:03:44.280 I am always tempted to fix those bugs that break existing programs.
00:03:52.560 However, breaking changes could cause significant pain to developers.
00:03:58.080 In the history of Ruby, we suffered from something called the Ruby 1.9 tragedy.
00:04:03.300 When I released Ruby 1.9, I had to make some compatibility changes to support encoding.
00:04:09.300 The changes included improved Unicode support, which introduced compatibility issues.
00:04:18.000 As a result, Ruby 1.9 and prior versions had a compatibility gap that affected our community for over five years.
00:04:28.500 Some community members continued using older versions, while we tried to improve the language.
00:04:35.640 This situation is tragic as some members became left out.
00:04:41.640 A similar situation happened in other languages.
00:04:48.660 We have seen communities split on issues like Python 2 and Python 3.
00:04:56.100 In the PHP community, developers tried to create PHP 6 but lost compatibility, leading to its cancellation.
00:05:02.140 They then restarted with PHP 7 while keeping compatibility to PHP 5.
00:05:10.800 Similarly, ECMAScript 4 introduced drastic changes.
00:05:15.960 They lost compatibility with earlier versions, leading to its cancellation and the introduction of ECMAScript 5.
00:05:21.840 They gradually added new features while maintaining compatibility.
00:05:29.340 This has proven that breaking compatibility does not work well with the community.
00:05:34.700 Programmers love new things, yet they equally despise the pain caused by those changes.
00:05:42.060 This is why compatibility matters.
00:05:48.240 We have learned that we must make progress without breaking the past.
00:05:54.860 New features should be designed with compatibility in mind.
00:06:01.019 Except for a few bug fixes, programs should be able to rely on consistent behavior.
00:06:07.680 I must confess, Ruby 3.0 did not maintain 100% compatibility due to necessary adjustments to keyword arguments.
00:06:14.360 We had to address the confusion surrounding keyword arguments.
00:06:20.580 This introduced some minor incompatibilities, which is unfortunate.
00:06:27.900 However, previous behavior contained design bugs that we needed to fix.
00:06:34.920 Apart from these corner cases, we strive to keep compatibility with Ruby 3.0.
00:06:42.960 Moving forward with performance matters.
00:06:50.640 As I mentioned, we faced fragmentation over five years.
00:06:58.140 The Python community faced similar challenges and suffered for an even longer period.
00:07:06.300 The difference lies in the size of our respective communities.
00:07:12.120 Ruby 2.0 brought significant changes along with many compatibility issues.
00:07:19.560 We introduced a new virtual machine, known as YARV (Yet Another Ruby Virtual Machine), which is considerably faster.
00:07:29.640 The community was forced to migrate to Ruby 1.9, which added speed.
00:07:37.620 The performance mattered a lot to users.
00:07:44.760 The willingness to adapt to cleaner language syntax comes secondary.
00:07:50.640 What we adore are the performance improvements.
00:07:58.080 Performance comparisons are popular.
00:08:05.700 However, micro benchmarks don’t define real-world performance.
00:08:10.980 Programming languages serve multiple purposes, from game development to scientific computing.
00:08:17.160 Measuring performance solely through micro benchmarks is misleading.
00:08:25.920 Ruby operates efficiently for most real-world applications.
00:08:31.620 For instance, Ruby on Rails software caters to robust services.
00:08:39.430 It may not be the absolute fastest, but it is certainly adequate.
00:08:45.719 Although performance matters, discussions often rely on micro benchmarks.
00:08:52.220 We are continuously working on enhancing Ruby's performance across the board.
00:09:01.000 This includes improvements for web applications, micro benchmarks, and everything in between.
00:09:10.440 For example, we have introduced several upgrades to our garbage collector.
00:09:19.760 We've implemented generational garbage collection in Ruby 2.1, then incremental garbage collection in 2.6.
00:09:28.460 Ruby 2.7 introduced transient heap garbage collection, enhancing memory management.
00:09:37.100 We also adopted object compaction to optimize memory allocation.
00:09:44.640 These improvements significantly enhance Ruby's performance.
00:09:51.920 Since Ruby 2.6, the introduction of the MJIT (Method Just-in-Time) compiler has further accelerated performance.
00:09:57.720 This compiler dynamically generates native code, enhancing speed.
00:10:02.440 One of Ruby 3's goals is to achieve "Ruby 3x3," meaning Ruby 3 is three times faster than Ruby 2.x in some benchmarks.
00:10:09.960 Particularly in the Ruby benchmark suite known as "Optcarrot," we have seen performance improvements.
00:10:15.680 For example, under Ruby 2.0, the benchmark function ran around 32.47 frames per second.
00:10:24.600 In Ruby 3.0, it reached approximately 44.46 frames per second without using MJIT.
00:10:32.480 With MJIT, Ruby 3.0 achieved 98.36 frames per second.
00:10:39.500 This demonstrates a performance improvement exceeding three times the previous versions.
00:10:45.660 MJIT provides substantial benefits to optimizing benchmarks, although it comes with some drawbacks.
00:10:52.000 It incurs compilation costs and can raise security concerns.
00:10:59.400 For instance, some servers prohibit compilers from being executed, which hinders our processes.
00:11:06.600 Furthermore, too many C functions can push instruction caps and hinder overall performance.
00:11:14.700 In JIT compilation, having too many methods can lead to instruction cache overflow.
00:11:20.820 We are continuously working on enhancing MJIT and overall performance.
00:11:26.640 In Ruby 2.6, Rails applications performed worse with MJIT.
00:11:34.420 Yet in Ruby 3.0, these applications run as fast as without MJIT.
00:11:41.140 We see promising improvements and continue striving for better performance.
00:11:48.100 Next, let's discuss concurrency.
00:11:56.300 The concept of concurrency relates closely to performance.
00:12:05.820 When I created Ruby in 1993, even the fastest computers had only a single core.
00:12:15.600 Today, with computers having multi-core processors, we must harness concurrent programming for efficiency.
00:12:23.060 Ruby has historically struggled with threads due to its Global Interpreter Lock (GIL), which limits true parallelism.
00:12:31.120 While it previously employed fine-grained locks, this approach slowed down Ruby performance significantly.
00:12:38.520 Thus, implementing true concurrency in Ruby has proven complex, especially with existing paradigms.
00:12:47.620 To address this, we have introduced two new concurrent entities: Ractor and Async Fiber.
00:12:56.060 Async Fiber is designed for I/O-heavy tasks, utilizing a non-blocking approach.
00:13:04.100 Many programming languages, such as Python and JavaScript, implement async I/O via promises.
00:13:12.660 Consequently, Ruby 3 adopts asynchronous I/O with fibers, enabling developers to create non-blocking code effectively.
00:13:21.080 Ractor, on the other hand, is intended for CPU-bound tasks.
00:13:28.060 It promotes isolation, with communication executed via channels.
00:13:34.320 This isolation enhances safety and prevents data races, allowing multiple Ractors to process independently.
00:13:42.020 An example of this would be deep recursive calculations.
00:13:47.500 The sequential version handles the recursion without isolation.
00:13:53.640 In contrast, Ractors can run concurrently, increasing performance.
00:14:00.060 This parallel execution makes Ractors an excellent solution for CPU-bound tasks and enhances overall Ruby performance.
00:14:08.560 We continuously look to improve concurrency as we progress.
00:14:15.660 In addition to enhancing performance, we are focused on static analysis.
00:14:21.480 Ruby 3.0 should catch more errors earlier, allowing for safer coding practices.
00:14:29.280 We are in an age where dynamic typing is morphing towards static typing.
00:14:37.260 Languages like PHP and Python are adopting static typing features to enhance reliability.
00:14:46.680 Even JavaScript has introduced TypeScript, a statically typed alternative.
00:14:53.640 The Ruby community has remained steadfast in its vision of dynamic coding.
00:15:00.900 Though we may not aim for full static typing, we should strive for better error detection.
00:15:06.420 To achieve this, we introduced RBS (Ruby Signature) and TypeProf.
00:15:12.840 RBS serves as a type declaration language resembling Ruby's syntax.
00:15:19.140 This will be pivotal in providing greater clarity and coherence in code.
00:15:27.660 We also have TypeProf, a tool to generate RBS data for your applications.
00:15:36.560 Though the journey has just begun, we are optimistic about the contributions this will make.
00:15:44.060 As Ruby 3.0 ships, RBS will be included for core and standard classes.
00:15:50.400 This allows RBS to assist type checkers and improve your developer experience.
00:15:57.900 Several VS Code plugins exist to assist with RBS in more efficient ways.
00:16:02.820 We are continuously enhancing Ruby's static typing capabilities.
00:16:08.620 Besides static typing, Ruby 3.0 has introduced new syntax.
00:16:12.960 Pattern matching was introduced in Ruby 2.7 and has since been improved.
00:16:20.340 Now, developers can retrieve data from structural hierarchies effectively.
00:16:27.900 Ruby 3.0 also embraces new agile features like one-line pattern matching.
00:16:35.820 Additionally, we added the ability to specify multiple symbols within arguments.
00:16:41.940 This creates easier and clearer code, enhancing overall readability.
00:16:49.020 Overall, we will continue to evolve Ruby towards a better future.
00:16:57.480 Although some syntax changes are coming, we maintain stability and consistency.
00:17:04.440 We will refine these changes and seek feedback from our developers.
00:17:12.840 That is crucial for us as a community moving forward.
00:17:18.780 Lastly, on a broader note, there is a need for improved support tools.
00:17:26.640 Supporting tools enhance coding experiences significantly.
00:17:33.960 We envision tools that bolster our language, elevate user contributions, and improve ease of use.
00:17:41.820 We are excited to integrate language server protocols to assist with type checking.
00:17:50.400 All these improvements ultimately contribute to a strong Ruby community.
00:17:58.680 I encourage each of you to contribute to sustain and grow this community.
00:18:05.640 Together, we will craft a better programming experience for all.
00:18:13.740 Thank you for joining me today, and I look forward to collaborating moving forward.
00:18:20.340 Now, let me take any questions.
00:18:25.860 Anyone? Please feel free to ask.
00:18:33.220 One question, will RBS support inline syntax, or is that too complex for Ruby's grammar?
00:18:42.360 I'm not currently planning to provide inline syntax for RBS.
00:18:51.300 Instead, I believe tools created by third parties can generate RBS from documentation.
00:18:58.680 In the future, I want Ruby to become smart enough to infer types without explicit declaration.
00:19:05.740 This approach can lead to a cleaner syntax with less clutter.
00:19:14.000 Any more questions from the audience?
00:19:22.200 I'm excited to hear your thoughts!
00:19:32.820 Please do not hesitate to ask.
00:19:39.300 Without performance constraints, what features would you love to see in Ruby from other languages?
00:19:48.740 I would love to introduce a packaging system.
00:19:58.100 Languages like Java and Python have more coherent module systems, which could benefit Ruby.
00:20:08.280 While structured packaging might be useful, developers want simplicity as well.
00:20:16.740 We should strike a balance and ensure imports don’t clutter our code.
00:20:25.040 If there are no more questions, I will conclude my session.
00:20:34.820 Thank you all for your engagement and participation.
00:20:45.080 Feel free to explore and learn more about Ruby features post-session.
00:20:54.980 We are excited about the future of Ruby.
00:21:04.680 Whether it’s through new tools or enhancements, the prospects are immense.
00:21:14.780 Let us all contribute and foster collaboration.
00:21:21.340 Thank you for being here today, and I hope to see you all in future discussions.
00:21:30.440 Goodbye, and until next time!
00:21:45.319 Hello everyone and welcome back. Here is live Mats, welcome!
00:21:52.559 Hi, thank you for having me! Thank you for coming.
00:21:58.980 There has already been a lot of hype about you being here.
00:22:04.920 We are super glad to have you despite the video quality issues.
00:22:12.540 It happens; nothing to be sorry about.
00:22:18.780 Everyone loved your talk anyways.
00:22:26.220 So, we were asking if there are questions in the chat.
00:22:31.740 Luca has one; let’s see what he says.
00:22:37.859 Luca commented that monkey patching is prohibited.
00:22:46.079 I’m not going to prohibit monkey patching; rather, be cautious with its usage.
00:22:52.319 It could potentially disrupt other team members.
00:22:58.740 What are your thoughts on this?
00:23:06.240 Yes, Ruby is like a sharp knife. Active support is convenient but can lead to complexity.
00:23:14.210 Special actions should remain obvious and contained.
00:23:21.340 Refinements offer a more modular way to enhance existing classes.
00:23:30.799 It's essential to maintain clarity while coding; let the code speak for itself.
00:23:38.600 Are there any more questions?
00:23:42.480 If there are no questions, are there any last minute thoughts you wish to share?
00:23:52.200 As Ruby evolves, the freedom it provides is essential.
00:24:00.960 However, developers must be responsible and considerate of their impact.
00:24:10.680 Everything has two sides, and one can enjoy freedom while also respecting maintainability.
00:24:22.480 We shouldn't take away developers’ freedoms but encourage responsible use.
00:24:29.960 Ruby's future lies in collaboration and proactive contributions.
00:24:37.660 There are plenty of ongoing projects to enhance our experience.
00:24:50.480 Please stay engaged as we continue to refine Ruby.
00:24:59.960 I am excited for what lies ahead!
00:25:07.960 If there are no further questions, I will conclude.
00:25:17.200 Thank you all! I appreciate your participation and enthusiasm!
00:25:32.220 Goodbye!
Explore all talks recorded at rubyday 2021
+2