00:00:04.600
Hello everyone, we're here to talk to you about building command-line applications for everyone.
00:00:10.000
I am not Yehuda, nor am I Godfree, but we are members of the Ruby community, and we're here to share our insights with you.
00:00:21.480
Today, we will discuss how to build command-line applications that work for everyone, while using Ruby.
00:00:58.519
Let's dive right into it.
00:01:08.159
Hi, I’m Terence. I go by h02 on Twitter. I wear blue hats and work at Heroku managing the Ruby experience.
00:01:14.920
That means when you push Ruby apps to the Heroku platform, that’s what I work on.
00:01:20.840
I'm originally from Austin, Texas, where we have great barbecue and amazing tacos.
00:01:26.720
Additionally, I'm one of the co-founders of the Ruby karaoke movement, and we're actually organizing a Ruby karaoke event tonight.
00:01:33.880
If you want to sing and haven't signed up yet, please go to the RubyKaigi events page to find the link and sign up.
00:01:45.680
I'm excited to see you all there! Today, we'll be discussing building command-line applications in Ruby.
00:01:52.079
This talk will be divided into three parts: firstly, we'll cover the history of packaging in Ruby, and why mruby is a fitting solution for these challenges.
00:02:06.000
Then, we'll discuss the work Z and I have done to create an excellent experience using mruby for building CLI apps.
00:02:21.760
Let's start by examining the packaging landscape in Ruby.
00:02:27.440
The Heroku tool belt provides an illustrative example to discuss this topic.
00:02:32.640
Initially, when we began six or seven years ago, Heroku was a Ruby gem, primarily serving Ruby customers.
00:02:41.159
This made it a good fit since it offered a native experience that Ruby users were familiar with, but it also meant we didn’t have to tackle packaging or distribution issues.
00:02:54.120
However, as we aimed to attract other types of users, such as Python developers, it became problematic.
00:03:06.480
Forcing others to install Ruby just to use Heroku was inconvenient.
00:03:12.319
Even for existing Ruby users, it created significant maintenance challenges, as we could not guarantee which Ruby versions our users had.
00:03:24.799
Consequently, we were often limited to supporting older versions of Ruby, which hindered our ability to introduce new features.
00:03:36.760
To solve this issue, we introduced the Heroku tool belt, bundling the Ruby gem with a statically compiled Ruby runtime.
00:03:47.480
This allowed us to deliver a package that could be built for all target platforms.
00:03:55.400
However, the process came with its challenges, requiring specialized machines for compiling different versions.
00:04:01.760
We had to use a Mac Mini for OSX, a separate Windows machine, and releasing became significantly harder.
00:04:08.599
To address these packaging problems, we explored other technologies.
00:04:19.759
One of my coworkers started rewriting the entire experience using Go, citing performance reasons as a significant factor.
00:04:37.120
For example, running a command to display the Heroku version took approximately 1.8 seconds in C Ruby, while it took only 16 milliseconds in Go.
00:04:48.240
So, the performance gains from switching technologies were substantial.
00:04:55.760
Moreover, Go provided a cross-compiling toolchain that eliminated our need for numerous physical machines.
00:05:01.080
This meant we could generate a single binary to distribute to all customers, a much more efficient solution.
00:05:07.560
This trend isn't unique to Heroku; companies like HashiCorp also transitioned many tools written in Ruby, like Vagrant.
00:05:12.800
While Vagrant is still written in Ruby, other tools they develop are being re-written in Go.
00:05:20.000
Another significant project was Shoes, designed to build desktop applications across multiple operating systems using native bindings.
00:05:34.440
This project, however, was challenging to maintain, leading the team to rewrite it in JRuby.
00:05:41.520
The maintainers noted that packaging issues were primarily what made the project's C code difficult to manage.
00:05:50.080
More recently, the Traveling Ruby project emerged, which was built by the creators of Phusion Passenger.
00:06:06.840
They aimed to package a precompiled Ruby runtime along with the application code and third-party extensions.
00:06:15.800
However, the issue with this approach was that third-party extensions had to be precompiled for each platform.
00:06:25.720
Moreover, when security fixes were necessary, the maintenance burden was significant.
00:06:32.760
This led to many solutions falling short.
00:06:39.920
Our goal is to continue creating Ruby applications without letting the technical limitations of packaging in MRI hold us back.
00:06:50.080
Today, we’ll discuss how mruby can help us solve these problems and why it’s a viable option.
00:07:02.080
The advantage of mruby is that it’s a smaller, embeddable Ruby that has been designed with a low memory footprint in mind.
00:07:09.120
This means we do not have operating system-level features like file or socket functionalities, or thread support.
00:07:17.639
Still, we can utilize the core of Ruby for building command-line applications.
00:07:24.639
You can still use procs, blocks, and other Ruby features to create versatile applications.
00:07:30.840
Even though mruby might not have as rich a standard library as CRuby, it still retains the essence of Ruby.
00:07:39.840
Interestingly, some projects have already employed mruby in production environments.
00:07:51.120
One example is the project Engine X, which enables extending it with Ruby.
00:08:00.320
We've recently implemented an Ember project called Dashboard, which runs on top of Engine X using mruby.
00:08:11.760
This is part of our largest web property, and it's currently running Ruby in a production environment.
00:08:24.160
Another area where mruby is gaining traction is in IoT devices, where resource constraints demand lightweight solutions.
00:08:36.480
In one case, a project called OpenWrt allows embedding mruby in routers for Internet access.
00:08:46.639
Users can set up configurations using a custom interface that determines which binaries are compiled.
00:08:59.280
All of this is achievable through using the mruby gem's build system.
00:09:04.760
Even though the core of mruby is small, the mrb gem facilitates access to numerous libraries.
00:09:18.960
The MB gem specification file is fundamental for any mruby application.
00:09:26.160
You can define dependencies much like you do with Ruby's gem specification.
00:09:35.479
There are essentially three sources to pull from for mrb gems.
00:09:50.520
The first source is the core mrb gems folder, which includes essential libraries.
00:09:58.200
Secondly, you can use the mgem source, which acts like a repository for publicly published mrb gems.
00:10:06.239
Lastly, you can fork projects or use GitHub directly to pull in any dependencies, even if they aren't published.
00:10:19.840
For example, we forked the mruby-o project while waiting for a Pull Request to be merged.
00:10:27.560
This allowed us to statically compile libraries that were critical to our project.
00:10:39.920
Today, we will delve into what’s behind the construction of these applications.
00:10:53.840
During this conference, we heard a talk by K SAS about adding a bytecode ahead-of-time compiler to MRI.
00:11:08.960
However, with mruby, we've already implemented something similar.
00:11:24.960
Mruby allows us to compile Ruby code into a binary format suitable for embedding.
00:11:39.360
Using the mrbc tool comes in handy to convert Ruby files into binary, enhancing performance.
00:11:50.960
This binary format can then be interpreted by the mruby interpreter.
00:12:03.240
You can embed this compiled binary code into a C program, which allows for a seamless integration.
00:12:16.560
After compiling mruby, you receive a build summary encapsulating all included gems.
00:12:23.360
However, there are limitations in this approach, especially concerning the embedding design.
00:12:34.480
Since mruby aims to remain small, various existing Ruby gems may require porting to function correctly.
00:12:46.320
We faced challenges with libraries such as op-parse that were not readily compatible.
00:12:56.000
Additionally, some frameworks did not work smoothly, leading to slower iterations during development.
00:13:05.440
Cross compiling for multiple targets also necessitates configuring the native extensions appropriately.
00:13:17.080
Recently, we added two flags to mruby’s build system to enable cross-compilation targeting.
00:13:27.639
This result was necessary to allow regular expression extensions to work in our Windows builds.
00:13:38.320
We navigated these configuration hurdles to ensure better user experiences when building applications.
00:13:47.880
That led us to create the mbCLI project, which Terence will discuss more in-depth.
00:14:03.200
One of our main design goals in the mbCLI project was to enable developers to write primarily in Ruby.
00:14:16.480
The mbCLI application itself is structured like any Ruby application, organized with multiple Ruby files.
00:14:31.520
Furthermore, just like the Heroku experience showed us, performance is indeed crucial for command-line applications.
00:14:42.680
Users expect quick interactions, so startup time must be optimized.
00:14:57.480
For instance, the simple hello world example takes approximately 41 milliseconds to run in MRI.
00:15:07.920
However, using the mbCLI we can drop that runtime to just 3 milliseconds, illustrating a significant performance difference.
00:15:18.239
Another major asset provided by the mruby ecosystem is the ability to create a single binary that runs across all major platforms.
00:15:28.960
In our case, compiling solutions for Linux, OSX, and Windows was essential.
00:15:41.040
We also aimed to keep file sizes manageable; for instance, the OSX mbCLI binary is only 421 KB.
00:15:52.679
Our goal is to build a solid platform for the Ruby ecosystem that developers can leverage to easily create command-line applications.
00:16:01.440
For adoption to succeed, we want to simplify the setup process, which is where Zach will take over and demonstrate creating a hello world application.
00:16:13.680
To get started with mbCLI, the first step is visiting the project page on GitHub to access the releases.
00:16:31.520
You will find various packages available for each architecture needed to run mbCLI.
00:16:40.640
As an example, while using a Mac, I downloaded the x86_64 Apple Darwin package, extracted it, and added the binary to my system path.
00:16:55.360
From there, I was able to output the version of mbCLI, confirming that it works as expected.
00:17:06.120
Additionally, the command can print help information, appearing just like any other command-line application.
00:17:20.480
We also have a setup command to generate applications for mbCLI.
00:17:34.240
All you need to do is pass the name of your application, much like using the 'rails new' command.
00:17:46.320
Once that’s done, a list of files gets created, and we will review each file's purpose.
00:18:02.240
Among those files, the build configuration file orchestrates how the mruby compiler functions.
00:18:14.480
We have various target configurations defined for compiling on native Linux hosts, OSX, and Windows.
00:18:26.080
Specifically, each target tells Ruby itself how to compile the application.
00:18:41.120
Included in this file is a gem configuration method, specifying which gems belong to your application.
00:18:56.200
In the gem specification for your application, you'll define dependencies required to run your code.
00:19:10.720
Most critical is the 'bins' option, which directs mruby to generate a binary from your application.
00:19:17.120
Alongside this, a bootstrapper for your command-line app is generated, which is just a C file.
00:19:32.520
Although you may not need to read the bootstrapper, it sets up the environment for command-line arguments.
00:19:45.840
Additionally, we generate a skeleton for the main method that needs to be defined globally.
00:20:00.200
The testing structure includes two types of tests: bin tests and unit tests.
00:20:12.720
The bin tests provide an integration testing interface, while unit tests evaluate the implementation of your code.
00:20:27.520
To run your tests, you also get a rake file, which provides a range of useful tasks for compiling and running your tests.
00:20:41.120
This rake file works best when executed inside a Docker container designed to manage the build process.
00:20:56.520
The workflow utilizes Docker to streamline support and maintenance as we build for multiple platforms.
00:21:08.560
Essentially, with the Dockerfile and docker-compose.yml, we create an environment to run all necessary commands.
00:21:24.320
By using these tools, you can compile your projects with the necessary dependencies and cross-compilation support.
00:21:40.080
The first command to execute after setting everything up is 'docker-compose run compile'.
00:21:53.600
This command builds the application while mounting your local volume and creating binaries for all targeted platforms.
00:22:06.480
Once everything compiles successfully, a summary is printed, outlining the binaries and included gems.
00:22:19.320
When you're ready to use the binaries, you can locate them in the build directory.
00:22:31.920
You can find corresponding entries for your native operating system that can be executed.
00:22:45.440
After everything has properly compiled, you can initiate tests with 'docker-compose run tests', which runs both unit and integration tests.
00:22:59.200
This enables you to iterate and refine your development before releasing.
00:23:08.960
When you feel satisfied with your CLI application, you can publish the files in the M build directory as needed.
00:23:21.440
For example, for mbCLI, we typically publish it on the GitHub releases page.
00:23:42.000
We're excited about our journey so far; there’s a lot of work ahead to improve the process further.
00:23:51.520
One project I’m eager to develop is enhancing our build system to create a more flexible and maintainable environment.
00:24:04.320
Another goal involves curating a collection of gems specifically tailored for building command-line applications in mruby.
00:24:21.320
We want to draw inspiration from Rails, which did a great job at offering reliable platforms and curated gems.
00:24:36.160
We plan to start by focusing on commonly needed features, like option parsing.
00:24:52.200
Our aim is to create a cross-compiled ecosystem that works seamlessly across all major platforms.
00:25:07.760
We welcome everyone to participate! Please install mbCLI and explore it; get Docker configured if you haven't.
00:25:23.000
We encourage you to create your own applications and share them with us.
00:25:32.880
Please don’t hesitate to tweet at us; we’re excited to see what you build in Ruby.
00:26:00.520
If you have a question, feel free to ask.
00:26:57.200
When you run your test suite, does it need to rebuild the VM every time?
00:27:05.160
No, it recompiles only the changed files.