Introducing Rubyfmt

Ripper

Introducing Rubyfmt

Penelope Phippen

Introducing Rubyfmt

by Penelope Phippen

In the talk "Introducing Rubyfmt" at RubyConf 2019, Penelope Phippen shares the vision and development journey of Rubyfmt, an innovative code formatter for Ruby. Penelope, as the director of Ruby Central, emphasizes her connection with the Ruby community and establishes the importance of the project she will focus on going forward.

Key points discussed include:

- Introduction to Rubyfmt: Inspired by Go's gofmt, Rubyfmt aims to provide a fast and efficient solution for Ruby formatting without configuration options, focusing purely on formatting consistency and speed, akin to Unix tools.

- Comparison with Existing Tools: Unlike RuboCop, which offers extensive configurability and multiple functionalities (formatting, linting, etc.), Rubyfmt will concentrate solely on formatting, thus simplifying usage and ensuring reliable outcomes.

- Principles of Design: Penelope describes guiding principles for Rubyfmt's development:

- Do One Thing Well: Emphasizing the importance of simplicity and focused functionality, she contrasts Rubyfmt with RuboCop, which can be overwhelming in its complexity due to numerous configuration options.

- Correctness Over Simplicity: Rubyfmt aims for complete compatibility with Ruby implementations. It utilizes Ripper from the Ruby standard library to ensure accurate parsing, recognizing that achieving this level of correctness requires more effort.

- Speed is Needed at Any Cost: Highlighting the benchmark of performance, Rubyfmt targets execution in under 100 milliseconds for larger Ruby files. To meet speed demands, core components are being written in Rust, differing from Ruby's performance limitations.

- Development Timeline: Penelope indicates that Rubyfmt is still in development and may take another six months or longer to complete. She invites the community to follow her updates on Twitter.

- Conclusion: Penelope acknowledges the complexities involved in building Rubyfmt, stating that it is perhaps the most technically challenging project she has undertaken. She expresses gratitude to the Ruby community and other developer tooling teams for their foundational work. For further engagement, she encourages the audience to visit the GitHub repository dedicated to Rubyfmt.

Overall, Rubyfmt represents an approachable, efficient, and reliable solution for Ruby formatting aimed at enhancing the developer experience.

00:00:13.139 I think you all saw me this morning. Hi, how's everyone doing? I'm so happy to be here. This is the first time I'm giving a talk as a director of Ruby Central, and we'll get into that. But I have a few personal updates to talk about before we get too far into it. This is going to get real emotional. I'm really proud to be here.

00:00:32.189 So just to introduce myself, my name is Penelope, my pronouns are she and her. I’m trans, and I’m a woman. I really couldn't have done this without the Ruby community. I've been coming to these conferences since 2013. I owe my career, many of my best friendships, and so much of my life to the folks in this room and beyond. I am just so proud to be able to stand here in front of you today and say this.

00:01:52.149 As I mentioned, I'm also the director of Ruby Central now. I will be helping organize this conference and all of the conferences going forward. I will be the program chair for RailsConf, so please submit a talk to RailsConf; we'd love to have you. I'm no longer an RSpec maintainer, as I have officially retired from doing that. With the additional responsibilities I’m taking on, I just wasn’t able to continue. I want to thank John Rowe, who takes over now as the lead maintainer of RSpec, along with Myron Marston, Andy Linderman, and David Chiu-Linskey, who took the time to teach me how to work on the framework and really helped me become the Ruby developer I am today.

00:02:56.557 Rubyfmt is going to be my open-source focus for the next while, and that's the purpose of this talk.

00:03:02.600 So, let's get into it; what is Rubyfmt? Well, for the last couple of years, I've been programming not just in Ruby, but I've also been doing a lot of Go. If you've all worked in Go, you've probably encountered the gofmt tool. What gofmt does is allow you to make any change you want to a file. You can mess it up in any way and just save the file. When you do, the code will snap into place. For example, if I have some poor indentation, I can fix a syntax error, and upon saving the file, the code will automatically adjust the indentation.

00:03:41.920 What's important is how quick this process happens. The file I was editing was about 2,000 lines of code, and gofmt executes in around 25 milliseconds. For context, your typical screen refreshes every 16 milliseconds. The existence of this tool in Go is just amazing. It makes working with code feel great, and we don't have something like that in Ruby today. So, I set out to build a very fast code formatter in Ruby that behaves like a Unix tool. Essentially, we take source code, and the output will be better-formatted, consistent source code that executes quickly.

00:04:36.130 RubyFmt will have no formatting-related configuration options, just like gofmt, which helps ensure speed and consistency. It will consume input from standard input or file names and write to standard output or back to files directly. It also won’t be distributed as a gem, as there is confusion regarding the reliance on various binary and system components. Project-level isolation through gems simply does not work for this tool, and I don’t foresee an efficient way to distribute Rubyfmt as a gem.

00:05:08.479 In terms of the development cycle, I’m likely at least six months away, possibly more, and I will be tweeting updates. My Twitter handle is PenelopeZone if you'd like to find out when it's ready. I could finish here and that would be it, but I’d rather delve into why I’m taking on this challenge. This is a complex problem to solve, and I’m going about it in possibly the hardest way.

00:06:14.360 To illustrate this, let's talk about principles and trade-offs. Imagine you are crafting beautiful handmade furniture that can last for generations and be passed down through families. You’re likely to be slow and deliberate, ensuring that everything is sturdy and well-thought-out. You probably believe that this is the correct and good method to construct furniture. Conversely, most people opt for cheaper options that can be assembled quickly but lack durability. Despite their inferiority in quality, such options aren't necessarily bad and can be functional.

00:07:06.919 There’s a tension here between principles and practicality. The same goes for software development. For example, I once firmly believed that Test-Driven Development (TDD) results in superior code. If you asked me this three or four years ago, I would have adamantly supported this claim, asserting that those who don’t practice TDD are subpar developers. However, after discussing this on a podcast, I’ve recognized that TDD can be difficult to teach and, if misapplied, may lead to more harm than good.

00:07:36.249 This topic reflects the trade-offs involved in building software—a discussion of what we need to accept to create what we’re aiming for. When I was designing RubyFmt, I realized that what differentiates it from existing tools isn't just technical effectiveness but my unique brand of care. I've distilled these guiding principles into clear statements: do one thing well, correctness beats simplicity, and speed is needed at any cost.

00:08:02.600 Let’s start with 'do one thing well.' To frame this, I want to discuss RuboCop. I have sometimes criticized RuboCop, and I want to apologize for that. It's actually a good tool and discussing it helps clarify its context. The RuboCop documentation lists an overwhelming number of categories of cops—formatting, linting, etc.—that can be configured, making it a highly versatile tool. However, RubyFmt focuses solely on formatting, allowing me to hone in on achieving top-quality Ruby formatting.

00:09:22.649 RuboCop is infinitely configurable, which forces teams to determine which configurations to adopt. In LA at RubyConf, Justin introduced a tool called Standard, a subset of RuboCop with a default configuration considered to be suitable for most Ruby teams. The idea is that discussions about configurations often take longer than the benefit gained from them. My intention with RubyFmt is to create a formatting solution that can appeal to a wide audience, if not to everyone's taste.

00:10:04.259 To further illustrate this, Justin's presentation ended with him wearing a burger hat, humorously endorsing Standard as RuboCop with only the best parts. The key takeaway is that RubyFmt is a different class of tool designed to focus solely on formatting without the overhead of complex configuration.

00:10:38.340 If you visit the RuboCop page, you’ll find a lengthy list of configuration options. Each combination of options presents a significant challenge in ensuring the tool remains practical and useful. Therefore, by doing one thing well, we embrace a singular approach that prevents the complications that arise from excessive configurability. RubyFmt will support one consistent style without variance.

00:11:27.000 The next principle is correctness over simplicity. When considering how to parse Ruby, a common tool is called Pazza. Its output is user-friendly but lacks complete compatibility with Ruby implementations. As a result, Pazza may not handle every Ruby program perfectly. While 99% of the time, this is adequate, there’s always the chance of failure in edge cases, which is unacceptable for a tool that modifies your code.

00:12:29.000 An alternative is Ripper, a component of the Ruby standard library that offers complete Ruby parsing. However, Ripper requires booting a Ruby interpreter each time, introducing delays that make it impractical for this use case. The output from Ripper can also be convoluted and difficult to navigate compared to that of Pazza.

00:13:05.200 RubyFmt leverages Ripper for compatibility, which entails significant effort in comprehending and implementing Ripper correctly. The choice to use Ripper stems from the need for absolute correctness, despite extra work being required to achieve a precise and well-functioning formatter.

00:16:36.210 Finally, the principle 'speed is needed at any cost' is crucial. GoFmt executes in just 25 milliseconds even with large files. In contrast, the fastest Ruby can execute an empty string command is about 25 milliseconds, indicating RubyFmt will not utilize gems. The speed of execution is critical for a tool integrated into users’ workflows, and the benchmarks we're targeting are under 100 milliseconds for larger Ruby files.

00:18:33.859 After attempting to implement RubyFmt entirely in Ruby, I've determined that the Ruby language's performance cannot meet the demands for speed required by the formatter. Consequently, I began rewriting core components using Rust, which offers the speed necessary for formatting Ruby code effectively. The goal is to ensure RubyFmt is quick and reliable.

00:20:08.049 In conclusion, while building software, I consider the principles that shape the software's design. In particular, when creating RubyFmt, I recognize that it must meet these principles to be successful. It's a complex challenge, possibly the most technically intricate software I've constructed to date. Nonetheless, I’m driven and motivated to ensure it meets the expectations of the Ruby community.

00:21:29.480 I want to express sincere gratitude to the RuboCop and Sorbet teams for creating exemplary Ruby developer tooling. Their advancements have laid the groundwork for projects like RubyFmt. If you want to explore RubyFmt further, please visit github.com/PenelopeZone/rubyfmt. Thank you so much for your time.