00:00:14.830
Hello Mike, are you working? Hi, hello everyone.
00:00:20.300
I'm here to talk about building CLI apps for everyone.
00:00:26.029
If you were looking at the board and saw something about managing a community for Ruby Association, this is not that talk. Unfortunately, he couldn't make it. So, yes, I'm Terence Lee, I go by @homu02 on Twitter. I come from Austin, Texas.
00:00:38.840
Austin is only an hour and a half north of here, and we have some awesome tacos. If you're ever in town, I'd be more than happy to take you out for some tacos.
00:00:51.290
I know some people have mentioned that there's also a Torchy's Tacos in San Antonio, which is halfway between here and the airport. I've heard they have awesome tacos.
00:01:05.420
I'm here with a few fellows, including Davey, who is not attending RubyConf. We run this weird conference called 'Keeper Weird,' which is in Austin, Texas. It just happened last month, so stay tuned for year three next year.
00:01:17.150
You probably heard her talk about how great or terrible the conference was. He was there as well. I'm also a big part of Ruby Karaoke. If you did not go to the karaoke event at RubyConf, you missed out on a really awesome and magical experience.
00:01:29.329
This was us singing 'Bohemian Rhapsody,' which is a classic. I was inspired by Charlie Nutter last year during RubyConf in Taiwan, where he karaoke'd for a marathon of seven hours. I managed to do about five.
00:01:42.710
It was a great experience. This is him singing with Plexus, which is always a good time. I must give a shoutout to PJ Haggerty, who cannot make it here, but he has been a huge proponent of Ruby karaoke and has helped organize many of them.
00:01:57.159
So much so that I actually organized none of them this year, but there has been one at every Ruby conference I've attended, which has been pretty great. I work at Miroku where I do Ruby things with Richard Neiman, who is sitting over there.
00:02:19.010
He was the one who got that picture on the first slide, drinking that medium-sized margarita. Everything's larger in Texas, apparently. Now, let's move on to the actual talk.
00:02:38.120
I’m here to talk about packaging in Ruby and why we should be building things in Ruby. To get a sense of how this works, I’ll discuss the Heroku Toolbelt, its story, how we started, and where we are today.
00:03:00.500
When we first started, we were just a Ruby gem. To get started, you just ran 'gem install heroku.' I think some people today are still using the Heroku gem, even though it's been deprecated. At the time, it was a great decision because we were targeting Ruby customers, which was the only thing supported on the platform.
00:03:21.470
Plugging into the Ruby Gems ecosystem made it easy to get up and running. We didn't have to deal with package management or other complexities because Ruby Gems had a specification, and if you were a Ruby developer, you already had RubyGems installed to build your Rails applications.
00:03:41.690
This allowed us to move quickly without many hassles. However, the downside that we learned over time was that it required people to have Ruby installed on their systems, which became an issue as we moved to become a more polyglot platform.
00:04:02.209
I’ve talked with other developers who create projects that required Ruby to get started, like the Sass project, which required Ruby for its functionality. This posed a huge barrier to entry for front-end developers who might not be familiar with getting Ruby up and running.
00:04:22.760
If you’ve ever helped out at Rails Girls or RailsBridge events, you are familiar with how painful this process can be.
00:04:36.979
Even if you were a developer, we couldn't guarantee that the version of Ruby running was the same one for everyone. We had to support various versions across OS X, which had Ruby 1.8.7 as the default version for a long time.
00:04:53.810
Even after Ruby 1.8.7 was deprecated and no security updates were being added, we still had to ensure our gem was backwards compatible. This meant we couldn't take advantage of any new features or syntax, which made debugging and maintenance quite a challenge.
00:05:04.550
So we moved on to essentially a packaged installer where we would package the Ruby runtime into what we called the Toolbelt. We took the gem and the runtime and had to build that for every platform we wanted to support.
00:05:20.630
I don't know if you've tried to package Ruby for Windows, but it's not very fun. We had a special box for that, and for OS X, we even had a Mac mini set up under someone's desk at the office just for packaging Ruby for OS X.
00:05:35.450
We’ve automated a lot of that since then, but it is still a lot more work compared to just updating a gem. If you're just updating a gem, you can run a simple rake release or some other task, and it's straightforward to get versions and updates out.
00:05:54.680
However, with the new Mac OS X signing requirements, we now need to have certificates and other prerequisites in place. Then, an internal project was started by an employee rewriting the entire CLI in Go, and one of the primary motivating factors was speed.
00:06:21.620
Previously, when we benchmarked running 'heroku version' on the Ruby gem or the Toolbelt, it took almost two seconds. The Go version, however, took merely 16 milliseconds. You can see this massive disparity in performance.
00:06:46.160
Midley, the Go version did a lot fewer tasks than the Toolbelt version, but it shows a significant performance baseline. A lot of this can be attributed to the overhead of loading dependencies at runtime within Ruby.
00:07:04.099
The 'require' statement can be quite slow, leading to performance impacts, especially in command-line applications. You want to split up your application as it grows because you don’t want a massive codebase to sift through for a single command.
00:07:24.139
This means that the larger the CLI application gets, the more hacks you've had to implement to only load the files you need at the right time throughout your code base.
00:07:41.990
The second attractive aspect of Go is that it allows you to statically link each binary, which means we can build a single file distributable for every operating system we want to support.
00:08:01.969
This means for Windows, OS X, and Linux, we have one file to package up and send to someone without a bunch of folders and runtime and environment variables to set up.
00:08:15.020
When we had an existing Ruby install to account for, we had to ensure we weren't conflicting with existing setups. I don’t think the Heroku Toolbelt story is unique. HashiCorp has a similar story with their Vagrant project.
00:08:40.430
Much of Vagrant was originally written in Ruby, but they transitioned from being a Ruby gem to packaging itself. All their newer CLI functionality is now written in Go for many of the same reasons.
00:09:08.930
If you do want to package stuff up in Ruby, there’s a project called 'Traveling Ruby' by the Phusion guys, probably best known for ModRuby and Phusion Passenger.
00:09:44.270
The benefit here is that you don’t have to rewrite your entire application to use Ruby, as it handles the packaging of the Ruby runtime and any native extensions you might use.
00:10:02.060
However, the downside is that you are limited to the runtimes and extensions they have pre-compiled, which can be an issue if you want to use something else.
00:10:26.960
When I was in Portugal, I talked to SF Eric about the Crystal project. Besides Go, Crystal is a statically typed, compiled Ruby-like language that allows for building applications.
00:10:42.400
Over the past few years, we have seen many Ruby applications or Ruby-based CLIs emerge, such as option parsers and tools like Thor.
00:11:07.320
It’s a bit disappointing to see many of them transition away from Ruby. I believe a big part of this shift is due to our packaging sore spot, as deploying a Rails application still involves Chef scripts that clone your repo, run Bundler, and set everything up.
00:11:30.050
I wanted to continue building things in Ruby, which is the inspiration for all this discussion. I’ve given a version of this talk at Rocky Mountain Ruby, and Steve’s response on Twitter highlighted that.
00:11:53.150
There’s nothing wrong with using other languages like Go and Crystal, both of which are excellent in their own right. If you are happy using those languages, feel free to continue.
00:12:12.690
However, I want to continue building in Ruby and find a way to address weaknesses in the Ruby ecosystem so that those who want to choose Ruby can do so.
00:12:38.060
I started a project with Zachary Scott called 'Move CLI' early this summer, and we spent a good amount of time addressing the packaging problem.
00:12:57.879
When writing with EmbRuby (em Ruby CLI), most of your code should be in Ruby. Inside of em Ruby CLI, you have a lib directory where you can have numerous .rb files.
00:13:23.580
You can organize your code in any structure you prefer as long as it resides in that folder. Performance is also a key feature, which we've learned from the Heroku story.
00:13:41.200
For command-line applications, how quickly can the runtime boot up? When I did my tests with Ruby 2.2.2, it took 40 milliseconds just to boot.
00:13:58.130
In contrast, a simple hello world application in em Ruby was able to accomplish this task in only 3 milliseconds. This significant difference provides considerable headroom for interesting developments in projects.
00:14:17.820
As the CLI apps expand, I expect the performance differential will only get larger. The third design goal we had was to produce a single binary for shipping.
00:14:38.510
Having a straightforward system to package everything, avoiding complex setups with environment variables, is an attractive feature.
00:14:57.400
Within em Ruby CLI, we produce a build for Mac, Windows, and Linux through one command that generates a single binary, making it user-friendly.
00:15:14.960
The Mac binary is only 421 kilobytes, which is not terribly large. For this to gain traction, the setup process must be straightforward.
00:15:38.660
To simplify the setup, we're leveraging Docker. If you're unfamiliar, Docker is a Linux containerization method that allows for a user-friendly environment.
00:15:56.100
We pushed a tag to Docker Hub containing all tools needed, meaning you don’t have to have Ruby or a C compiler installed.
00:16:13.520
This means you can focus on one platform while still allowing cross-compilation for various others.
00:16:29.660
When you run the simplest hello world example with em Ruby CLI on the path, you set it up similarly to a Rails project by naming it, executing `cd`, and running a compile task through Docker.
00:16:50.379
Once compiling is finished, you can run your created binary to see the output, showcasing a smooth workflow beneficial for developers.
00:17:12.490
Now, let's go through what the build process generates and how all the components fit together. Em Ruby CLI serves as a Rails-like template generator to streamline this whole process.
00:17:32.240
To understand how it operates, we first need to delve deeper into em Ruby itself. Em Ruby is an embeddable Ruby interpreter designed for lightweight and quick start-up.
00:17:55.010
It specifically adapts to different architectures, enabling fast boot times. Many are curious about where it’s being utilized.
00:18:12.290
For instance, in Japan, a company uses em Ruby to extend router firmware, allowing developers to write code without delving deep into C or assembly.
00:18:26.300
Em Ruby is built for various architectures, including Android and iOS, and even IoT projects like Raspberry Pi and Arduino. However, it does not guarantee a full file system or common OS-level features.
00:18:43.500
This means no sockets or threading capabilities since they rely on operating system-level functionality.
00:19:01.360
The syntax is a subset of Ruby 1.9, with some features from 2.x, but it remains true to Ruby's language principles.
00:19:19.900
To run em Ruby code, you can generate an em Ruby binary—allowing you to execute scripts directly, giving access to Ruby’s built-in features.
00:19:36.060
Additionally, em Ruby supports a bytecode compiler. You can compile any em Ruby script into a .mov bytecode format, enabling efficient execution.
00:19:54.520
With the obtained bytecode, you can run the interpreter with the appropriate flags and eliminate the compilation step.
00:20:10.890
Taking it further, you can embed your code within a C file, utilize the compiler, and produce a single static binary.
00:20:26.240
To facilitate use, we’ve created a wrapper script that interacts with the compiled code, allowing developers to work primarily in Ruby.
00:20:44.450
To execute their code, they simply define a single method called '__main__' that represents the entry point for their Ruby code.
00:21:06.290
This structure allows seamless integration into the Ruby environment, focusing purely on Ruby code without delving deep into C programming.
00:21:25.700
Within em Ruby, the entire build system itself utilizes Rake, meaning developers can compile and test using normal Ruby tasks.
00:21:48.120
One of the unique aspects is the build configuration file generated as part of the package creation process, pre-filled with necessary details.
00:22:02.680
You don’t need to write everything manually unless you have specific requirements or target platforms.
00:22:20.010
Ever Ruby gems are akin to RubyGems in functionality. With an emphasis on a smaller footprint for CI development, we only want to include essential dependencies.
00:22:41.240
We maintain a clean distinction in package sources too, categorizing accordingly to ensure dependencies are correctly handled.
00:23:04.890
This helps keep the environment lightweight, balancing essential features with the requirement of using only what's necessary.
00:23:18.430
Unit testing in em Ruby works well with some limitations, as does integration testing using the existing MRI framework, allowing for thorough validation.
00:23:39.870
However, you should be mindful that not everything in MRI is directly translatable to em Ruby. The standard libraries are not as rich, and not all functionalities are available.
00:23:58.520
In creating cross-platform tools, we face challenges with dependencies that require native extensions. Solutions must cater to all target platforms.
00:24:15.450
One success story includes the JRuby launcher, which allows the JRuby ecosystem to maintain a unified command base across operating systems, adapting it with usability in mind.
00:24:39.050
My colleague Joe Kutner has worked on a project called MJ Ruby, rewriting the JRuby launcher in em Ruby, which made it more Ruby-friendly and transparent.
00:25:03.480
This experience illustrates the potential for Ruby to coalesce with other systems, including the JVM, while also making it approachable for developers.
00:25:27.220
Now, what can you do with em Ruby CLI? This ecosystem is relatively new, and the community is still emerging. It’s only about five years old publicly.
00:25:50.920
The project is more familiar to Ruby developers since it’s hosted on GitHub, enabling active collaboration and interaction.
00:26:15.180
Stepping back, I also want to note that there have been recent developments to enhance stability for future versions of em Ruby.
00:26:35.540
You can download binaries for various platforms from the releases page, placing them on your path for ease of use.
00:26:58.870
To build a binary, you'll need to download the em Ruby CLI binary, ensure Docker is set up, and follow the steps to generate a new application.
00:27:18.070
Then, you can modify code as you like, attempt to innovate, and see how the cycle of modification and recompilation works.
00:27:39.330
I’d love to hear about the projects you're building, regardless of their size. Engaging with the community is vital, and I appreciate how collaborative we all are.
00:28:00.290
As we endeavor to create more things in Ruby, I hope we can help remove the technical limitations that have historically impeded using Ruby.
00:28:21.750
Let’s work together to develop impactful CLI applications that reignite interest in Ruby while cultivating collaboration.
00:28:42.220
Now, to address some questions about the bytecode spec. As far as I know, it’s not published, but Matt would be the better person to confirm.
00:29:01.250
The separation of dependencies and final binaries is managed in our testing setup, allowing tests to exist without being included in deployment.
00:29:23.330
Notably, the original builds are tailored to ensure you have debug options, which can help in both local and separated target builds.
00:29:45.610
The original Toolbelt does not integrate with the new structure heavily. If you are running a transition, sticking to existing implementations might be easier until then.
00:30:08.790
Finally, while the packaging looks somewhat similar to Ruby gems, there are significant differences, particularly in how dependencies are handled.
00:30:29.590
While you could theoretically create a package with both, compatibility would present considerable challenges. A bridge between the two environments is not currently straightforward.
00:30:52.590
Thank you everyone for joining in. I hope this has been informative and that you are excited to participate.
00:31:10.630
Let’s continue building interesting things in Ruby! Thank you.
00:31:22.330
Are there any additional questions?
00:31:24.560
Thank you! Let's keep the conversation going.
