Ruby LSP
Frontiers of Development Productivity in Rails

Summarized using AI

Frontiers of Development Productivity in Rails

Rafael França • September 26, 2024 • Toronto, Canada

In the keynote address titled "Frontiers of Development Productivity in Rails," Rafael França, a Rails Core member and principal developer at Shopify, explores enhancements aimed at improving developer productivity within the Ruby on Rails framework. Having contributed to Rails since 2012, Rafael shares insights gathered from his extensive experience in building and maintaining the framework, particularly with the recent release of Rails 8.

Key points discussed in the talk include:

- Rails Philosophy: Rails emphasizes programming happiness, guided by principles such as 'Convention over Configuration,' allowing developers to focus more on application logic than configuration hassles.

- Challenges in Environment Setup: Rafael uses the experience of setting up a Rails application as an example, highlighting common challenges developers face, such as managing Ruby versions and database configurations.

- Development Environment Solutions:

- Vagrant: Introduced as a reusable tool for automating virtual machine setups but noted as heavy and complex.

- Docker: Offered a modern approach to creating development environments but often required knowledge of Docker commands instead of traditional Rails commands.

- Dev Containers: Rafael introduces this new solution integrated with VS Code, which allows developers to build a container-based environment that mirrors local development while minimizing the complexities. This approach maintains an ideal balance between ease of use and productivity by making the setup process faster and more consistent.

- Future Directions: Rafael urges the Ruby on Rails community to invest more in productivity tools. He advocates for well-integrated CLI tools that do not depend on JavaScript/Node.js, improved editor integrations for a better coding experience, and exploration into static typing for Ruby, aiming for better code analysis and suggestions.

In conclusion, Rafael emphasizes the importance of continually advancing the four foundational pillars of Rails and urges the community to innovate further in developer productivity. The vision is to reduce setup complications in Rails applications and enhance the experience for developers at all levels.

Frontiers of Development Productivity in Rails
Rafael França • September 26, 2024 • Toronto, Canada

Rails is known to be one of the best frameworks in terms of empowering developers to build great products, and has kept this place for 20 years. But...can we do better? In his talk at #RailsWorld, Rails Core member Rafael França shows how they are pushing Rails to continue making developers lives easier across new frontiers.

#RubyonRails #Rails #Rails8 #productivity #devcontainers #lsp #developerhappiness

Thank you Shopify for sponsoring the editing and post-production of these videos. Check out insights from the Engineering team at: https://shopify.engineering/

Stay tuned: all 2024 Rails World videos will be subtitled in Japanese and Brazilian Portuguese soon thanks to our sponsor Happy Scribe, a transcription service built on Rails. https://www.happyscribe.com/

Rails World 2024

00:00:12.840 My name is Rafael França. You can find me on Twitter (X) and GitHub at Rafael França.
00:00:19.800 I'm originally from Brazil, specifically from the state of Ceará, which is the smallest state in Brazil, located right on the coast. It's a sunny, warm tropical paradise. But now, I live in Canada, so you can imagine how different it is for me, especially in the winter.
00:00:27.840 I've been a member of the Rails Core team since 2012. For those who don't know, this team is responsible for developing and defining the direction of the framework. It's not just one person working on this; it's a collective effort.
00:00:39.559 Since 2014, I have held the first position in the list of all-time contributors to the framework. Thank you!
00:00:47.280 I’m also the person who released Rails 8 today during this keynote. I work at Shopify, which I believe you all know.
00:00:54.440 At Shopify, I am a principal developer in the Ruby on Rails infrastructure team. I helped to create this team, where I am responsible for setting the direction of Ruby and Rails within the company.
00:01:01.440 You might know this team because we recently created various projects, such as the new Ruby parser, the Ruby package manager, and many other contributions to the Rails framework and the entire ecosystem.
00:01:12.600 But enough about me. We are here to talk about Rails.
00:01:19.560 Rails has a very defining doctrine—a collection of pillars defined by David Heinemeier Hansson that has driven Rails since its first version.
00:01:26.840 Some of these pillars are things that the community is really proud of. For the purpose of this talk, I'll focus on a few of these pillars.
00:01:34.080 First, we pride ourselves on the fact that Rails is optimized for programming happiness, not for machines. Working with Rails should bring joy. The principle of "Convention over Configuration" means that you can focus your time on writing your code without needing to configure different libraries to work together.
00:01:46.840 This is because Rails has made a lot of choices for you, so you can just focus on what’s important for your application, and Rails takes care of ensuring that those pieces work together.
00:01:55.879 These four pillars are what attracted me to Rails, and they are the main reason why I'm still here. Rails excels in these areas.
00:02:02.280 Let's imagine ourselves in the minds of a beginner. We want to create a new Rails application. As we heard earlier, we visit the Rails website and start reading the Rails guides.
00:02:09.879 In the guide, we learn that we need to run 'rails new.' However, a section before that tells us that we need to install Ruby first.
00:02:18.680 How do we install Ruby? I mean, it should be easy, right? I just need to find the proper way to start Ruby on my operating system. Let's say I'm a Linux user because I heard that the creator of Rails has a newfound passion for Linux and has an amazing setup.
00:02:25.960 I am familiar with Linux; I know how to use it. I use APT to install packages, so it’s just a matter of running 'apt install ruby.' But wait a second: which version of Ruby? APT suggests Ruby 3.2, but I've heard that the new version is 3.3. So how do I install a different version than the one that comes with my operating system?
00:02:39.000 I search around and find information about something called version managers: RVM, rbenv and ASDF—it's a letter soup, and I'm really getting confused about which one I should pick.
00:02:47.280 How many of you use CRuby? Raise your hands. I see a lot. How about RVM? A lot of hands. And ASDF? Okay, many. How about rbenv? That's so little.
00:02:55.280 Okay, I'll go with the majority and use RVM. I finally install Ruby and can create my application, after I don’t know, a few hours of work.
00:03:02.120 I trust the Chef's choice and just create a Rails application with what’s provided to me. I run 'rails new,' but the Chef's choice sets the database to PostgreSQL.
00:03:12.400 However, I already know how to use MySQL; I've been using MySQL for years with PHP. So I don’t want to learn a new language and a new database at the same time.
00:03:19.680 I delete my app and search online, finding that to create a Rails application for MySQL, I need a specific flag. I pass the flag and generate a new app.
00:03:26.680 Oops! The installation fails. I don’t know why. It seems the system is missing some packages, but which ones? I search around and read the error message.
00:03:35.400 I learn that it’s related to the MySQL client that I just changed. I need a library to be able to install the MySQL client, so I run 'apt-get install' and all goes well.
00:03:44.640 My setup is properly configured, and I can happily build my app after a few hours of trying to create my first application.
00:03:51.760 After some time, our little app is now a business. While I could work alone on it because it's a one-person framework, I need a new team member.
00:03:58.680 I tell my new team member to clone the application, but now they need to set it up in the same way that I did. But again, it's been a few months, and I don’t remember what they did.
00:04:06.840 We are back to the same issue: how do I show Ruby? I used a version manager last time, but this time my friend is on a Mac. How do I set RUBY on macOS?
00:04:14.960 I search around and learn about Homebrew. I use Homebrew to install RVM, and I can now use Ruby again.
00:04:22.480 Great! However, it fails again. Why does it fail? It’s again the same problem: the MySQL client package is missing.
00:04:30.320 I know the name of the MySQL client package, so I try to install that package only to find out that the package does exist in this OS; it’s just called MySQL.
00:04:39.720 Again, I need to find out, and finally, I can run the tests.
00:04:47.640 My tests fail because I forgot that I was using Redis, which is probably the wrong choice, but I chose it anyway.
00:04:54.640 I forgot about it, so I have to start over, and finally, the application is working on another person’s machine.
00:05:03.920 I take notes for the next person so they don't have the same problem, and we can finally work on our product.
00:05:09.480 So you are going to be the judge now of how good we are in those four pillars.
00:05:17.160 Looking at this, we just learned that Rails is supposed to bring your application from an idea to an IPO. But how do you get to that point?
00:05:27.560 I’m sure many of you have had similar experiences, and in my humble opinion, that’s why we are good at these four pillars.
00:05:37.680 After you have your application, we should be better. We are not doing enough; we need to do more.
00:05:45.680 And that's what I'm here to talk about.
00:05:52.120 What are the frontiers of developer productivity in the framework we love, and what else should we be doing?
00:05:59.680 Much of what I talk about today will focus on environment setup—how you improve and set up your machine to develop your programs, which is essential before working on the application itself.
00:06:06.800 Some applications require minimal setup—you can just show Ruby, and that's fine.
00:06:12.400 But many others have very complex setups that require other dependencies. There are many ways to solve this problem.
00:06:20.800 I'll describe a few of them, starting with the first solution I heard about to the last one.
00:06:27.600 The first one is the exclusion of many bash scripts that people create to set up their computers.
00:06:36.840 We had a project—well, we still have a project called Vagrant, a way to automate the creation of virtual machines for development.
00:06:43.920 This is what a Vagrant file looks like. The language used to configure Vagrant should be familiar to us because it's based on Ruby.
00:06:51.600 A Vagrant file defines which operating system the virtual machine will use, what type of network it will use, and which ports we want to forward to our host operating system.
00:07:01.040 Additionally, it provides details about how much memory and how many CPUs should be allocated.
00:07:07.600 A shell script can also define all the dependencies you want.
00:07:14.640 Vagrant is great because it's reusable. You can just get this Vagrant file and share it with your colleagues, and they should be able to use it.
00:07:20.560 The environment is disposable; if anything fails, you can just delete the VM and create it again.
00:07:27.200 It’s repeatable; you should always get the same results given the same input.
00:07:34.200 However, since it’s based on virtual machines, it's heavy. You also need to think about data sharing between the host machine and the VM.
00:07:42.520 Boxing was another solution for the same problem created years ago by GitHub. Personally, I tried to use it.
00:07:51.040 It was a macOS-specific provisioning project that used a series of Puppet modules to manage the development environment.
00:07:57.960 I won't bother you with the details of how this works, but suffice to say that it was complex with its own DSL.
00:08:06.640 Disclaimer: I was one of the maintainers of Boxing at the time. I thought maintaining Rails was not enough of a full-time job, so I took on this complex open source project.
00:08:16.080 Boxing was local, as you could run it in your operating system, making it reusable like Vagrant.
00:08:25.880 However, it was not repeatable since it depended on the current state of your machine. If you already had MySQL installed, it sometimes worked and sometimes failed.
00:08:32.560 It was macOS specific, so it couldn't run on Linux or Windows. It was quite complex because it utilized Puppet.
00:08:39.600 With the advent of containers, people started using Docker to set up their development environments.
00:08:49.440 To illustrate this kind of setup, let’s take the Rails Contributors application—a real open-source application that we actually use.
00:08:56.640 This app has a defined Docker Compose configuration with two services, defining the application and the database.
00:09:06.800 In the application, you define where it is mounted, and you also set up where the database is located.
00:09:12.840 In the Dockerfile, you would list the Ruby version—for example, Ruby 3.3, along with all the packages you need to install to develop this application.
00:09:21.760 Packages may relate to your setup, like VIM, and others specific to the application itself, like the Push client.
00:09:30.240 Things were mixed, and I could just run commands like 'docker-compose up' to install Ruby.
00:09:39.360 This image shows how this works: you have two boxes: one is your host machine and the other is the container.
00:09:46.400 You would have the database in the container while the application would run there; however, your editor, browser, and files are on your local computer.
00:09:53.760 You would need to send explicit commands to the container, so you could not just run 'rails server' because Rails isn't running on your machine.
00:10:02.080 You would have to send the Docker command to run Rails in that container.
00:10:08.560 If you've been paying attention to Rails, we released a similar idea a couple of years ago, a new tool called Dock D.
00:10:16.080 It was a way to run Rails applications inside Docker containers.
00:10:24.000 Like any of these approaches, this Dock environment had its advantages and disadvantages.
00:10:31.680 It's reusable, disposable, repeatable—much like the Vagrant setup—but the abstraction leaks.
00:10:38.640 You need to know about Docker. You cannot run your commands in the usual way; you need to talk to the container to run those commands.
00:10:46.560 You also had data synchronization issues that could slow down your environment. Running 'git status' might take seconds because of file synchronization.
00:10:53.280 The latest solution to this problem is called Dev Containers. Much like Boxing, it is a suite of scripts to set up your development environment.
00:11:01.440 The advantages are similar to those of Boxing and Vagrant; Dev Containers are reusable because you can share them with colleagues.
00:11:10.080 However, they are not repeatable if you already have a machine set up. If you mess something up and want to reset everything, you may need to format your OS.
00:11:18.000 Although it is the newest in Rails 7.2, I am working on a feature called develop containers for Rails.
00:11:26.720 But wait a second: didn’t I just talk about containers in Docker? Why is this a different topic for the same thing?
00:11:36.040 Yes, that's true, but developing containers (or Dev Containers for short) are more than just a Docker setup.
00:11:42.560 It's a specification from Microsoft supported by many tools like VS Code, RubyMine, and it is even used to implement features like GitHub Codespaces.
00:11:52.240 Unlike the previous approaches, you develop inside the container, so the experience is the same as if you are running everything on your host.
00:12:01.760 To understand how this works, let’s take the same Rails Contributors application. This app has a folder called 'devcontainer' with three files inside it: devcontainer.json, docker-compose.yml, and Dockerfile.
00:12:11.360 In the devcontainer.json, we define the name of the application we are using, the compose file with the services required for development.
00:12:21.760 In this case, the application and a predefined setup for what features we want inside the container.
00:12:29.040 For instance, I want the GitHub API inside the container, I want NPM installed, and I want the Push client in the compose file.
00:12:38.320 I define the application itself and the database similarly to the Docker example but point the application to the Dockerfile.
00:12:46.760 We pass the version of Ruby we want to use, while the Dockerfile specifies which packages are needed to run the application.
00:12:57.760 I know that I will need the Git repository inside the container, so I clone the Rails repository.
00:13:08.000 Let’s see this in action! I'm not going to play the entire video, but we go to VS Code, tell it to clone the Rails Contributors application, and specify the main branch.
00:13:15.840 It starts to load the dev container. You can see what's happening in the logs; it creates a lot of containers.
00:13:25.840 In less than three minutes, we will have a real application set up. In this case, I don't even have Ruby installed on my computer.
00:13:35.320 So, to put this explanation in a diagram, a container is made of images, such as the Ruby image and a few additional features.
00:13:44.440 For example, we might connect to PostgreSQL and have a working Node.js client with some customizations.
00:13:51.680 In terms of personalization, you could clone your dotfiles inside the container so that you have your git aliases and bash scripts organized.
00:13:59.920 The image itself is just a composition of numerous features, like the Ruby image. The only thing that's there is the Ruby feature running in the right version.
00:14:10.720 This architecture supports a very high level of customization because we can create containers with many components, like having an application's own setup.
00:14:20.200 Now, let's dive into one of the features: the Ruby feature. Inside it, we have two files: a devcontainer feature, and an install script.
00:14:28.400 The devcontainer feature just defines what the feature is and what customization it has.
00:14:37.440 For example, this feature for VS Code will install the Ruby LSP extension. If you have VIM, it would install any additional requirements for it.
00:14:45.760 It also sets the path and exposes an environment variable for you so you know where to find the RVM command.
00:14:53.600 You can also define all dependencies for the features you want to install—say, a specific version of Ruby.
00:15:02.960 Inside the install script, it knows what packages we need to install Ruby and manages it for you.
00:15:10.320 For example, it handles any prior dependencies you might not think about, like the need to install GCC, to compile some gems.
00:15:19.760 It also clones RVM and installs it in your Ruby path.So you don’t have to worry about that.
00:15:28.000 The Ruby image simply consists of a Dockerfile, and in the Dockerfile, you define where the base Docker image is found.
00:15:34.640 You specify which features you require, such as Git and Ruby itself.
00:15:43.760 If you pay attention to the configurations, the Ruby image configuration looks similar to the dev container configuration we created for the application.
00:15:54.560 That's by design; Dev Containers are simply images, allowing us to build specific images for applications.
00:16:02.640 Instead of asking each of you to build the image on your machines, we can do it on CI and provide you with a built image specific for your app.
00:16:10.080 This means instead of taking three minutes or five or more, it will only take the time needed to download the image.
00:16:18.000 So how do I use this in Rails? If you have Rails 7.2, you can create a new Rails application and pass a Dev Container flag.
00:16:27.440 If you have an existing Rails application in Rails 7.2, you can run the 'rails container' command, and it will detect what you are using.
00:16:35.520 It will also generate the right dev container for you. But again, to run those commands, I need Ruby. How do I start Ruby?
00:16:43.520 To quickly generate the Dev Container configuration, if I use VS Code, I simply need to install the Dev Container extension.
00:16:51.280 You will see all the commands that the Dev Container provides in the command palette.
00:16:59.440 You can open the current project in a Dev Container or clone the project repository.
00:17:06.960 One of the tips I have for you is that if you clone the repository inside the container volume, not on your machine but inside the container volume...
00:17:15.760 You will have operations way faster, as we mimic the setup talked about earlier.
00:17:24.360 If you clone on your machine, you will have to deal with the file synchronization challenges that slow things down.
00:17:31.440 Much like the Docker approach, the pros are largely the same, but the challenges exist in the setup.
00:17:41.760 Perhaps you're thinking, 'Okay cool Rafael, but I don't use VS Code.' All the examples you gave are with VS Code.
00:17:49.920 Since Dev Containers were created by Microsoft for VS Code, does that mean they only work with it? Not really!
00:17:59.600 Since it’s a whole specification, you can already use Dev Containers with various editors. I’ve tried some myself: VS Code, IntelliJ, Emacs, and more.
00:18:06.880 But you still need Docker, right? Not necessarily. You can use it with any container runtime.
00:18:14.720 The ones I've tried included WSL (Windows Subsystem for Linux) and Cola.
00:18:24.320 So how about Ruby? I still need to start Ruby to generate a new application. To solve this problem, we created a standalone executable.
00:18:33.360 This works on any operating system, including Windows, and creates a container to generate the application for you.
00:18:40.000 This allows you to get a Rails application configured with a Dev Container that you can start in your favorite editor.
00:18:47.480 I'm not comfortable with this just yet because I think we need to invest a little more in it, but it works.
00:18:58.560 And that's exactly why I'm giving this talk: because I believe we should take these four pillars of the doctrine seriously and invest more in this area.
00:19:06.760 Dev containers and Docker are solutions for the environment setup problem, but they are not nearly the experience I want to give to people.
00:19:15.160 The CLI requires a JS runtime, for example, so to run Dev Containers you need Node. I’m not fond of that idea.
00:19:22.840 That’s why I want the community to try to build a CLI in any language, like Rust, that does not require JavaScript.
00:19:34.920 Additionally, we could invest more in this area. I think editor integration is crucial; we could do more as a community.
00:19:42.640 We have the Ruby LSP in the Stimulus LSP right now, but we need to enhance it. We need better code suggestions and timely information where it matters.
00:19:50.360 Why should I wait for running my Ruby program to know that there’s a syntax issue? That should happen while I write.
00:19:58.560 Since I’m already discussing this, we should also talk about static analysis of code. I'd like us to examine static typing seriously.
00:20:07.840 I understand that none of the incarnations of this for Ruby are perfect yet, but I would love for us as a community to investigate what an ideal static typing system would look like for Ruby.
00:20:15.840 We as a community need to keep pushing the frontiers of productivity further and further.
00:20:26.320 After all, this is an area where we pride ourselves on being good at.
00:20:34.480 I hope I was able to spark a bit of curiosity about the subject, and I look forward to seeing what our community will create in this space.
00:20:41.920 Thank you!
Explore all talks recorded at Rails World 2024
+17