00:00:00.240
It is time to introduce our next speaker, Soutaro Matsumoto. Soutaro is a lead Ruby engineer at Square, working on static typing for Ruby. He is a core Ruby committer and designed and implemented RBS for Ruby 3. He also developed Steep, a static type checker for Ruby.
00:00:11.599
The integrated development environment, or IDE, is one of the most frequently used tools for programming. It’s a kind of text editor where you type and read the code, but it does more for you, including on-the-fly error reporting, completion, go-to-definition, and more. These features help you write and read the code, making the tools more valuable than simple text editors.
00:00:39.440
Soutaro has been working on IDE development to support Ruby programming with a static type checker. It is based on the Language Server Protocol (LSP) and implemented in Ruby. He wants to share his experience with us today. What is the protocol? How can the LSP features be implemented? You will gain insights into the under-the-hood workings of IDEs, making the tools more familiar to you.
00:01:10.159
Now, let's start with his presentation. All questions can be directed to the stream chat, and Soutaro, it's your turn.
00:01:23.840
Okay, so here I am. This is my first time attending Euruko, and I am very excited to present today. You may notice that my family name is the same as Yukihiro Matsumoto's. You might be wondering if we are related, but actually, we are not. Matsumoto is a really common surname in Japan. Let me introduce myself a bit further.
00:02:19.000
I work at Square, and I live in Tokyo, Japan. I am one of the Ruby committers and have been working on type checking for Ruby, which was released as RBS last year. I also developed a static type checker called Steep, which is based on RBS.
00:02:25.840
My talk today will not be directly about RBS or Steep but will focus more on the development of IDEs. However, before I start, I quickly want to introduce Steep. It is uncommon to implement an IDE in Ruby, but you can install the tool using RubyGems or Bundler.
00:03:01.440
Initially, it was developed as a command line tool, so you can run type checks on your projects with the Steep check command. It will read all the Ruby source code and perform type checking, reporting any errors to the console.
00:03:28.640
However, relying solely on a command line tool is not a great user experience. You write some Ruby code in a text editor, switch to the terminal, run the Steep check, and then return to the editor to fix any errors, which can be cumbersome.
00:03:54.160
To improve this experience, we developed an IDE plugin that integrates with Steep. I developed a Steep extension for Visual Studio Code, allowing you to install the extension from the marketplace. This plugin assists with reading and writing code by providing on-the-fly diagnostics reporting, completion, and navigation between different parts of your code.
00:04:19.680
In this talk, I will focus on what an IDE is and the features it provides. For instance, Visual Studio is a very established IDE, having existed since approximately 1995. In fact, Visual Basic was my first programming language, and I wrote code on Windows using an IDE and did some debugging, among other activities.
00:04:41.920
Of course, one of the most popular IDEs today is VS Code, which I have used for writing Android applications for 10 years. We should also mention the fantastic IDEs released by JetBrains. I really love RubyMine, as it is one of my primary development environments. Other examples include Xcode for iOS development and Android Studio for Android.
00:05:06.960
You might say that Emacs is an IDE; indeed, you can do almost anything in Emacs. Interestingly, I found a funny phrase on the website of Vim, which states that many consider it a complete IDE. Thus, it's apparent that discussing which IDE is better can lead to heated debates.
00:05:41.760
Every IDE has its strengths. There are numerous features that help you read and write code, compile applications, configure settings, run tests, and debug your applications. Most IDEs provide graphical user interfaces to manage these activities more efficiently.
00:06:03.440
In this talk, I will specifically focus on the text editing features of IDEs. These include syntax highlighting, code folding, diagnostic reporting, navigation, and completion features—elements that significantly enhance the coding experience.
00:06:40.960
For instance, I have a screenshot of Visual Studio Code showing its syntax highlighting and folding features. You can see symbols in the left column of the editor, allowing users to fold and collapse specific sections of the Ruby code, making it easier to navigate through large codebases.
00:07:10.399
Diagnostic reporting is also essential when editing code. In the IDE, as you work, the program analyzes your code in the background. If any issues are detected, they will be reported immediately in the editor, allowing you to address them without needing to switch to another interface.
00:07:36.480
Hovering over certain elements in your code can provide instant documentation. For example, if your mouse pointer hovers over a variable or method call, it will show relevant documentation or type information. This feature is implemented by Steep.
00:08:02.379
Navigation features allow users to quickly jump to definitions within their codebase. By clicking on parts of the code, you can easily navigate to the methods or types being referenced, which enhances the coding workflow.
00:08:28.080
Completion features are also incredibly beneficial. When you start typing, the IDE automatically suggests completion candidates, alleviating the burden of typing out lengthy names. Screenshot examples of this functionality highlight how the IDE supports various refactoring tasks, such as renaming methods or extracting code into new methods.
00:09:15.360
However, IDE features are often dependent on an in-depth understanding of the programming language itself. For instance, implementing syntax highlighting or code folding requires knowledge of the language's grammar and syntax. This means that features such as error reporting, navigation, and others must be built on top of rigorous program analysis.
00:09:53.760
Before continuing, I want to outline some levels of program analysis. Some analyses are simple, while others are complex. Text-based analysis is the most fundamental type of analysis; it treats input as a sequence of characters without understanding their meaning.
00:10:21.680
Running an input function may check for issues like forbidden characters or overly long lines of code. More advanced analyses, such as syntactic analysis, understand the structure of the code and can implement features like syntax highlighting or diagnostics.
00:10:53.680
Semantic analysis is one of the most complicated forms of program analysis, as it involves understanding the types of variables and how they are used throughout the program. This level of analysis is necessary for advanced features like navigation, completion, and refactoring.
00:11:23.680
As I continue my talk, it’s essential to understand that developing IDEs is notably challenging. IDEs require their own program analyzers because different technologies and languages often have varying APIs, requiring separate implementations for features common across IDEs.
00:11:47.620
A potential solution is to extract common analysis functionalities from IDEs and use the Language Server Protocol (LSP) to facilitate communication between IDE front-ends and backend analysis engines, thereby reducing the need for individual implementations.
00:12:08.620
The LSP was designed by Microsoft and released five years ago. It allows for communication between development tools and language servers through a protocol designed for interprocess communication, meaning language server implementations are not limited by their underlying technology.
00:12:56.120
Currently, LSP is supported by many popular text editors, including Visual Studio Code and Sublime Text, making it a widely adopted technology. Consequently, IDE development is improving, with language-specific features being developed alongside shared analysis engines.
00:13:29.760
Numerous language servers for Ruby exist, such as Steep's language server and Stratum, which is another Ruby language server. Steep was developed as a static type checker but supports the Language Server Protocol. My presentation will focus primarily on the implementation of Steep.
00:14:18.920
The architecture of Steep involves various IDE front-end tools communicating with Steep's language server via standard input/output. When events occur, such as text document changes, the IDE sends notifications to Steep, which processes them, performs type checking, and sends any errors back to the IDE.
00:15:06.239
For example, when a user types a new character, a notification is sent from the IDE to Steep in JSON format. The server then recognizes the change, updates the source code, and begins type checking. After this, any resulting type errors are communicated back to the IDE.
00:15:47.520
The server-client communication structure can be visually likened to that of a web server and browser, but LSP allows messages to be sent back from the server to the IDE, such as publishing diagnostics to inform users of errors as they code.
00:16:15.039
Here’s a simplified version of the language server implementation. The server receives events and checks if they indicate a change in text documents. If so, it updates the relevant source code and conducts type checking. However, a naive implementation may encounter responsiveness issues.
00:16:46.720
To ensure the language server remains responsive, I implemented some optimizations, the first being incremental type checking. This approach allows the server to focus on the changed file rather than re-checking the entire project, leading to faster type checking.
00:17:19.600
For instance, if there are 100 Ruby files and you make changes to only one, we can skip checking the other 99 files, which dramatically improves performance. This method involves a fast pass, where only the affected files are type-checked.
00:17:55.020
Conversely, if changes are made to files that define types and interfaces (such as RBS files), it may take longer to type check, as many functions depend on those definitions. This is similar to editing header files in C programming, triggering potential long re-compilation times.
00:18:35.760
To enhance user experience, I introduced prioritized type checking for open files. This strategy emphasizes checking files currently in the editor so that developers receive immediate feedback, facilitating a smoother coding experience.
00:19:12.920
Another key optimization involved dropping unrelated code during completion requests. Completion responsiveness is vital, as users will wait for potential candidates while coding. By filtering out methods unrelated to user input, we can speed up the search for relevant completions.
00:20:02.520
In conclusion, I have discussed IDE features that require advanced program analysis and how the LSP assists in developing those IDE features across various programming languages. I have been working on Steep as a static type checker for Ruby, and I shared insights into enhancing IDE responsiveness.
00:20:56.800
Thank you for your attention! That concludes my talk.
00:34:04.240
Oh, the joys of live events! As you said, Euruko 2021 is indeed lively. Thankfully, nothing alarming happened, and we can continue discussing the presentation.
00:34:24.800
The audience's response has been overwhelmingly positive, with many thanking you for the talk, and there's a question about which language server you would recommend for VS Code users.
00:34:45.200
My recommendation depends on what you're trying to achieve. If you're using Steep, it would be a great choice. Otherwise, Sorbet might be suitable, too. I personally love using Steep, as I develop features for the RBS gem.
00:35:16.160
There have been several compliments and interesting questions from the audience. For instance, one person expressed how your talk made them even more interested in LSP. I appreciate the community engagement and hope to continue the discussions on Discord or stream chat, as more questions may arise.