Technical Background of Interactive CLI of Ruby 2.7

by ITOYANAGI Sakura

In this presentation by ITOYANAGI Sakura at RubyConf 2019, the technical background of the new Interactive Ruby Shell (IRB) introduced in Ruby 2.7 is discussed, emphasizing its multi-line editing capabilities powered by the new Reline library. The talk provides insights into the historical context of terminal development and the challenges associated with character encoding and internationalization in programming tools.

Key Points:

Introduction to Speaker: ITOYANAGI Sakura introduces herself as a Ruby committer and RDoc maintainer, sharing her journey and the significance of Ruby in her career.
Adventure in Ruby: Sakura likens her work to an adventure, paralleling experiences in climbing with exploring unknown territories in Ruby development.
Overview of IRB Improvements: The updated IRB aims to enhance user experience with multi-line editing. This new feature was supported by the Ruby core team and was a vision of its original author, K.G. Sang.
Historical Context of Terminals:
- Discussion of the evolution of communication technologies, from Morse code to modern terminals, outlining key inventions such as typewriters and teletypes.
- Emphasis on the development of escape sequences, control codes, and their relevance to Unix-like systems and Windows.
Character Encoding and I18n Challenges: Highlights the complexity of character encodings globally, particularly for languages like Chinese, Japanese, and Korean.
- Explanation of Unicode and its role in standardizing character representation, using emojis to demonstrate code points and graphemes.
Technical Structure of the New IRB: The presentation underlines how Ruby handles different characters, emphasizing methods like String#codepoints and String#graphemes for managing character data effectively.
Demonstration of New Features: The talk concludes with a live demo showcasing the new IRB, focusing on its multi-line editing capabilities, while providing reassurance about legacy modes for compatibility with older versions.

Conclusion:

Sakura emphasizes the innovative strides made with Ruby 2.7's IRB, which integrates both historical insights and modern programming needs. The new features are set to improve the interactive development experience in Ruby, while being mindful of the diverse character needs of a global user base.

00:00:12.850 Let me introduce myself. I'm a Ruby committer and the current RDoc maintainer. I'm also a member of the Ruby core team. My name is ITOYANAGI Sakura, and I’m a member of Space Pirates.

00:00:21.170 Our business involves receiving funds from venture companies that commission software development to us. This company was funded by my friend two years ago, and we started with only five employees. My company has supported me as a semi-full-time OSS engineer.

00:00:39.530 As a Ruby committer, my hobby is climbing. Normally, I visit climbing areas before our international conferences. However, this time, I couldn't go climbing before RubyConf because I had to attend the Ruby World Conference as a speaker.

00:00:47.840 I thought about an adventure in Ruby. The adventure means going to places that nobody knows. It's a world that nobody understands the value of, and everyone lives in their well-known comfort zones.

00:01:07.250 For example, think about Steve Jobs. Adventure is not just one week after the Ruby World Conference presentation. I came here, and even though I couldn't climb around Nashville, I found a good place to climb nearby.

00:01:22.179 This is a world map. I'm from Japan, and Nashville is quite far from Japan. Puerto Rico is also located across the ocean. I tried climbing in an unknown area about the jungle of rhetorical, where I heard there are many alligators.

00:01:36.530 The idea of the unknown is important for adventures. Today, I will talk about my adventure in Ruby. I'm currently working on RDoc, which is a standard documentation tool for Ruby, and I’m trying to improve IRB, the Interactive Ruby Shell.

00:02:07.880 The brand new IRB features multi-line editing powered by Reline. This multi-line editing feature was advocated by K.G. Sang, the author of the original IRB, and the Ruby core team member. It’s a great vision, but it is also quite challenging to implement.

00:02:29.450 The original IRB is implemented in GNU Readline, which has over 30 years of historical background. Therefore, we need to ensure compatibility with many features from GNU Readline.

00:02:47.840 This improvement is structured into three sections. First, I will discuss the history of the terminal, which includes several developments: Morse Code, typewriters, teletype, escape sequences, and support on Unix-like operating systems and Windows.

00:03:06.560 Most of these communication technologies were invented over the last 60 years, and they have significantly influenced how we interact with terminals today.

00:03:19.050 For example, Japanese people have been cultivating rice for over 10,000 years. Rice served as a form of currency in Japan about 200 years ago. Merchants faced difficulties due to price variations across regions, needing a fast communication technology.

00:03:43.060 This led to innovation in communication methods, like the telegraph invented by William Cook and Charles Wheatstone. The telegraph served as a primitive means of transmitting information over distances using tapping sequences, similar to Morse Code.

00:04:56.060 Types were invented, and eventually, the teletype was developed—improving communication speed and efficiency.

00:05:04.780 The development of the typewriter necessitated the invention of ribbon-controlled paper. The operations required for typing were then adapted to modern terminal protocols, enabling character movements, line feeds, and carriage returns, which form part of today's control codes.

00:05:25.070 In the late 19th century, terminals separated keyboards from printers, forming the basis for modern printing systems. With the advent of computers, these terminals evolved further, with printers gradually replaced by visual monitors.

00:05:46.190 Around this time, new terminal software was being developed such as termcap, which encapsulated different escape sequences for improved compatibility across various terminals.

00:06:03.980 The introduction of control codes and their evolution through new technologies has brought complexity, leading us to where we are now with computer terminals.

00:06:33.180 The rise of the internet and the introduction of UNIX-like systems in the 1980s led to the terminal's further evolution. Users started demanding more from their terminals, and now both UNIX systems and Windows have their own unique characteristics in terminal management.

00:06:53.560 Today, terminal emulations involve a broad set of control sequences, which allow various formatting and control operations. However, this has led to a confusing landscape for software and user expectations across different platforms.

00:07:11.680 With the complex history of terminals, we approach the need to integrate compatibility and functionality into Ruby's IRB, taking lessons from both past technologies and modern requirements.

00:07:32.980 Now, I'd like to shift focus to the importance of character encoding and internationalization (I18n) support in contemporary programming and tools.

00:07:45.680 There are numerous character encodings around the world that create challenges, especially in complex languages such as Chinese, Japanese, and Korean. The significance of Unicode becomes paramount, as it attempts to unify how characters are represented across different systems.

00:08:03.970 In a world rich with symbols and combinations, understanding how this works, especially for non-CJK (Chinese, Japanese, Korean) people, can be challenging. Therefore, I will illustrate this using emojis.

00:08:23.610 Emojis are a nuanced feature of Unicode. The distinction between code points and graphemes is essential for understanding. For instance, some code points are invisible because they serve as modifiers for other characters.

00:08:47.670 Let’s look at the telephone emoji. Its representation may vary depending on whether it’s displayed in textual or emoji styles. This example shows how subtle differences can exist between characters and their various representations.

00:09:14.550 Similarly, national flags are constructed with regional indicator symbols (such as U and S for the United States) that demonstrate the complexities inherent in coding and displaying characters effectively.

00:09:32.740 Unicode holds a rich and sometimes confusing history, working to ensure that characters are human-readable while accommodating the intricacies of various languages.

00:09:48.150 Understanding how Ruby handles characters is vital for efficient development. The `String#codepoints` method retrieves code points, while `String#graphemes` manages graphemes. This is essential when dealing with strings that consist of multiple code points but represent a single grapheme.

00:10:08.150 Thus, removing just one code point from a grapheme can lead to issues in display, emphasizing the importance of managing character data correctly.

00:10:14.050 All of this constructs the technical background to understanding Ruby's interactive shell. The brand new IRB introduced in Ruby 2.7 will utilize these principles to provide an improved user experience.

00:10:35.090 I will release the brand new IRB before Ruby 2.7, so you can install it and start using it right away.

00:10:46.490 Thank you for your attention, and please look out for the new version of Reline as the first step into the future of Ruby interactive development.

00:11:07.070 Now I’ll proceed to finalize the presentation with a live demonstration of the new IRB, showcasing its features and capabilities.

00:11:21.380 Expect the process to be a bit slow as the installation commences, but I will guide you through it step-by-step.

00:11:34.810 The IRB version you are about to see is radically different from previous versions, and if you encounter any challenges, please remember that legacy modes are still available for older versions.

00:11:55.610 For instance, let’s define a function to illustrate how multi-line editing works in the new IRB environment.

00:12:12.280 We’ll create multi-line entries which demonstrate the improvements made. The live coding will showcase the capabilities and functionality of the new system.

00:12:34.560 As we proceed, I encourage you to form questions about Reline or the new IRB, as I will address them once we're done with the demonstration.

00:12:54.150 So, let’s begin the demonstration, and thank you for your patience.