Talks
Keynote: The Story of JRuby
Summarized using AI
Keynote: The Story of JRuby
by Charles NutterIn his keynote at Euruko 2017, Charles Nutter discusses the story of JRuby, a Java implementation of the Ruby programming language founded in 2001. He begins with the history of Ruby, which was created in 1993 but gained international exposure through key publications around 2000. Nutter shares anecdotes from the early days of JRuby's development, highlighting the contributions of notable figures like Tom Enebo and the development history with various core contributors over the years. Key points from the talk include:
- JRuby's Origins: JRuby started as a parser and has evolved significantly, now focusing on full compatibility with Ruby language specifications, achieving 98% compatibility.
- Development Progress: Over the years, JRuby has undergone numerous changes and improvements through contributions from hundreds of developers, with thousands of commits and issues managed via platforms like GitHub.
- Support for Modern Ruby: The current JRuby version (9.1) is compatible with Ruby 2.3, with ongoing development to ensure compatibility with Ruby 2.4.
- Native Extensions and Libraries: While support for native extensions was initially challenging, it's improved over time, with many popular libraries available for JRuby. Nutter mentions the significant advancements made in supporting Ruby on the JVM, benefiting from Java’s robust libraries like Scala and Clojure.
- Deployment Options: JRuby supports a Ruby-like deployment environment, allowing the use of servers like Puma and Java tooling for packaging applications, exemplified by the use of Warbler for creating self-contained applications.
- Performance and Optimization: JRuby leverages the JVM's optimizing capabilities for improved performance, implementing a native JIT compiler that significantly enhances the execution speed of Ruby code compared to CRuby.
- Concurrency and Parallelism: JRuby can utilize true concurrency and parallel execution, leveraging JVM capabilities to run multiple threads simultaneously, a limitation in CRuby.
- Community Involvement: Nutter encourages developers to get involved in JRuby's development and improvement through contributions, highlighting the openness to adding features both in Ruby and Java.
The session concludes with an invitation for developers to explore JRuby's capabilities and participate in its continued evolution, emphasizing the language's strength in merging the Ruby and Java ecosystems.
00:00:08.069
Okay, good morning everyone! Hello! We made it! You all survived the party—that's great. I'm glad to have such a good crowd here.
00:00:13.559
Let's get going; we have about 40 minutes and a lot of stuff to cover. So, I am Charles Nutter—Charlie to my friends. This is my first time in Budapest, and I'm having a wonderful time exploring, trying craft beer, and seeing all the sights.
00:00:27.270
I've done three Eurukos, but I have not done any of them since Amsterdam, so it's been several years. Before that, I attended one in Prague. So, this is the only three Eurukos I've really been excited to be back for.
00:00:40.740
I work on JRuby and am a co-lead of the JRuby project at Red Hat. I am very thankful for five years of sponsorship by Red Hat—their support keeps the project going and helps us work within the community. Today, we’re going to tell a little story.
00:01:03.870
Once upon a time, on the 24th of February 1993—how many people know that Ruby is actually this old?
00:01:10.979
That's right, 24 years! Yes, it’s been around for a while. Our wonderful friend, the man who changed the world by creating Ruby, started this journey back in 1993.
00:01:26.270
For many years, Ruby was pretty much just known in Japan. It continued to evolve and saw new releases, but it was not internationally recognized until around the year 2000, when two fellows, Dave Thomas and Andy Hunt—known as the Pragmatic Programmers—released one of the first books in the United States for the Ruby programming language, the original pickaxe book.
00:01:46.540
Around the same time, this gentleman, Jan Arni Peterson from the University of Bonn, became interested in Ruby. About a year after the book came out, he made the first commit for the first version of JRuby, and this is where it all started.
00:02:10.090
How many people knew that JRuby actually started in 2001? It’s much older than you’d expect! Yes, it's surprising; almost none of the original people are still on the project, and we've probably rewritten every single line of code since then. But JRuby has actually been around for quite a while.
00:02:36.310
So, what did we have in the beginning? JRuby started as essentially just a Java implementation of the Ruby parser, so it could be used for tooling, code analysis, and so on. Initially, it was really just a parser.
00:02:49.090
The original JRuby was just the parser and a few core classes. If you’ve ever worked with Ruby’s parser, you’ll understand that Ruby strings and Ruby arrays exist in the parse tree as well, so you need some of those pieces.
00:03:07.629
That was the very early beginnings of an interpreter, but it was primarily just focused on language processing. About a year later, my good friend Tom Enebo got involved. Here’s Tom doing one of his trademark faces! We travel around the world and check in beers together—this is me trying to make sure I have him in the background doing one of his sad fish faces. About a year later, Tom got his first patch into JRuby.
00:03:45.659
He’s been involved with the project even longer than I have! In 2002, as often happens with open-source projects, his first commit was just deleting a bunch of stuff, but it was a beginning. He got involved, he got interested in Ruby, and he wanted to make JRuby happen. Around this time, the interpreter started to fill out a lot more.
00:04:17.650
Now we had more core classes taking shape, and the structure of the project largely stayed the same to this day. The main JRuby package contains all the core classes, while we have compiler and interpreter packages underneath that.
00:04:44.639
By 2002, the core team included folks like Chad Fowler, who made small contributions to the project. This began to grow; it was about two busy years of releases, around 2001 to 2002. There was a little change in versioning because it was based on Ruby 1.6.
00:05:09.099
So, for a while, we had 1.6 underscore as the rest of the version. Throughout 2002, we continued to improve compatibility. Basic code was starting to run alongside basic integration with Java code and Java libraries, but it kind of went quiet for a while.
00:05:43.560
For about two years, there were no releases. Code continued to be added to the project—there were small improvements—but not really any major releases. Meanwhile, Ruby itself was moving forward, jumping to Ruby 1.8, which would form the basis for the Ruby community we know today.
00:06:08.889
The 1.8 series started to take off. The pickaxe book was being picked up at more and more conferences, and JRuby began to fall behind a little. There just weren’t enough folks working on it, and nobody was sponsored to continue the work.
00:06:29.289
Implementing Ruby takes a lot of work, so 2004 was a big year for Ruby. I had the pleasure of going to RubyConf 2004, where I got to see well-known Ruby figures like David Hansson present Rails for the first time, and I also saw Koichi Sasada present RVM, which became the basis for Ruby running today.
00:06:54.580
Jim Weirich was there talking about new projects such as Rake and RubyGems. During this time, we even held hack fests on the earliest versions of RubyGems and Rake. It was a very exciting time, especially compared to the Ruby crowds we have today; there were a grand total of 65 people at RubyConf 2004!
00:07:27.070
We all got to know each other; it was a lot of fun. It was the fourth Ruby conference in the United States, but the first one I actually got to attend. At the time, I was a Java Enterprise Architect. Exciting, isn’t it? I had reached the pinnacle of Java development.
00:07:47.500
The company I was working for was based in the Washington, DC area, so I traveled out there and, as luck would have it, RubyConf 2004 took place during the same time I was in town. I paid my $50 to attend the conference, not even really knowing what this language was.
00:08:12.790
A friend recommended that I check out Ruby, saying it was kind of cool and fun to play with, but I was completely in the dark. At every talk, every slide, every code example I saw, I could read and understand immediately.
00:08:35.860
This had never happened to me as a Java developer. I would go to Java talks, as a Java architect, and wouldn’t understand what I was looking at on the slides.
00:08:57.260
So, sight unseen, I was able to understand all this code, and I was amazed! But I was a Java guy—I had to find a way to use Ruby as part of my job.
00:09:20.080
Being from Minnesota, where half of the year we're under a layer of ice and snow, I started to look for any projects that had Ruby on the JVM. It turned out there was such a project; my old friend Tom was the current maintainer and pretty much the only person working on it at the time.
00:09:46.450
So, I got involved and started to make my own small contributions—again, as is common in open-source projects. I improved some build aspects of the project, and I started to work on it. It was a lot of fun! I spent my weekends and evenings trying to get things going and improve performance and compatibility.
00:10:09.360
About a year later, in 2005, I gave my first-ever presentation at RubyConf 2005 in San Diego. This was me doing the first JRuby talk anywhere, I believe—the first JRuby talk anybody had ever done at that point.
00:10:31.230
I walked through why someone would want to combine Ruby with Java, and I wasn’t quite booed off the stage, but there were definitely interesting comments asking why I would do such a thing.
00:10:52.370
Combining the beauty of Ruby with the horror of Java was a tricky proposition! This was, of course, in a time when Ruby was largely made up of people who were former Java programmers.
00:11:16.950
So, this was post-traumatic stress for them, to have me say, "I’m bringing Ruby to Java! You will never escape Java!" But it went well; the talk was well received by a number of folks and we managed to continue working on JRuby.
00:11:41.390
Fast forward, it’s been 11 years since I got involved in JRuby—11 years since we were hired to work on JRuby by Sun Microsystems, one of the first companies willing to sponsor our project and help us succeed.
00:12:10.660
We've had hundreds of contributors over the years, and we still have dozens that come and go on a regular basis. We’ve dealt with thousands of issues, and we’re on our second bug tracker now, having moved from JIRA to GitHub.
00:12:28.280
In both cases, we have thousands of issues that we’ve worked on—tens of thousands of commits! I think it was 43,000 commits to JRuby the last time I checked. And, of course, a huge thank you to all the companies that have sponsored us—including Red Hat, Engine Yard, and Sun Microsystems.
00:12:53.500
JRuby would not exist without this kind of sponsorship. Now, let’s get into JRuby itself. What is JRuby today? Well, JRuby is Ruby, first and foremost. It’s Ruby.
00:13:14.320
We don’t emphasize the fact that it’s Java or on the JVM. If all you want is Ruby code, that's all you need to worry about. We focus heavily on compatibility—98% of the core language specs pass.
00:13:39.390
The features that we don't have implemented are often unusual. For instance, how many people have actually used the flip-flop operator in Ruby? Only a handful, right? If you really want it, I accept patches; but it’s an unusual feature.
00:14:02.540
We just haven't bothered to implement it with our new interpreter. Currently, JRuby 9.1 is the release line, and if you’re going to grab JRuby, this is what you’ll get—it’s Ruby 2.3 compatible.
00:14:27.020
How many folks have moved to Ruby 2.4? All right, some folks are moving on—that's good! We have Ruby 2.4 compatibility pretty much done on master.
00:14:43.150
For those of you using Ruby 2.4, I encourage you to check out our nightly builds, see if things are working, and check for missing features. We've gone based on Ruby's tests to ensure our features are implemented properly.
00:15:06.360
Probably 99% of the tests written for Ruby 2.4 features pass now, so I think we're in pretty good shape! We’re putting some finishing touches on this and hopefully, we’ll have that released around RubyConf time in about two months.
00:15:28.430
One tricky area for us has been native extensions, but the JRuby ecosystem has matured significantly over the years. Almost all popular native libraries with C extensions for Ruby do have JRuby versions—there's Jason, Nokogiri, and so on. They're all out there with JRuby support.
00:15:54.470
Many others have pure Ruby versions that work well on JRuby; in fact, in some cases, JRuby can run Ruby code for an extension just as fast as MRI can run the C code. Usually, it’s not a problem; we can just use the Ruby version of it.
00:16:21.660
We’ve also been supporting the FFI library (Foreign Function Interface). We maintain the FFI for both JRuby and CRuby, which allows you to write Ruby code, load up a C library, and start calling functions on it—no extension required other than FFI itself.
00:16:42.870
Of course, there’s an entire library of Java classes and Java libraries available through JRuby, including Scala, Clojure, and Kotlin. I’ll show some examples of that later.
00:17:05.130
Rails is still the big hitter in the Ruby community. Rails 5 pretty much works today on JRuby. The main issue is that there was a large rework of how ActiveRecord worked internally, so we have lagged behind on ActiveRecord support.
00:17:29.030
We’re currently working on some new adapters that will finally get us full active Rails support. These new adapters are going to be much cleaner. About 75% of the code in the SQLite version, which is mostly done now, is just the code from Rails.
00:17:52.030
We’ve really reduced the footprint of what we have for custom code for JRuby—which is also due to the re-architecting they've done in ActiveRecord. They've implemented a better design that separates the database driver from the object relational mapping.
00:18:17.080
We’ve also worked to support previous versions of Rails with the same driver. Rather than checking 'if it’s Rails 3, do this; if it’s Rails 3.1, do this; if it’s Rails 4, do this,' that’s all gone! We don’t have to maintain that mess anymore.
00:18:41.930
This should make it much quicker and easier for us to keep up. We’re also going to work with Rails to get our drivers as part of Rails proper, so we can maintain it all in the same place.
00:19:01.829
We’ll know if things fall behind, and we can keep up with our compatibility level. The good news is that with the new version of our ActiveRecord adapter, we're in the 99th percentile for supporting Rails tests—this is by far the best we've ever been!
00:19:20.860
So if you’re doing Rails applications, we’ll get these adapters released in the next month or two, and they should work fine for your applications.
00:19:39.649
Now, if you have an application built in JRuby, how do you deploy it? I assume you have to use a Java application server, right? Well, no! Again, we focus on the Ruby way of doing things.
00:20:01.080
Servers like Puma work just as well on JRuby—better in some cases! The typical ways of deploying Ruby applications still work. We do have Java-style ways of deploying too.
00:20:20.560
For example, TorqueBox is a server that wraps a Java Enterprise set of services with nice Ruby interfaces and a way to bundle it all together. We also have Warbler, which allows you to take an entire Rails application with JRuby and all its dependencies, put it in one file, and run it.
00:20:40.230
You can simply throw it on a server using 'java -jar your_file.jar', and you've got an application running—it's by far the simplest way to package a Ruby application. We have integration with Maven and Gradle, along with other Java tools.
00:21:01.530
So if you're familiar with the Java side of that ecosystem, JRuby fits pretty well into it. If you’re interested in deployment, I strongly recommend checking out Joe’s book—he did a great job covering many ways of deploying applications.
00:21:21.560
From typical Capistrano sorts of deploys to war files, TorqueBox, Heroku, and more—it's a good read! The other half of JRuby is that we are a JVM language, which gives us advantages on the Java platform.
00:21:42.350
We benefit from better garbage collection, native JIT, parallel threads, and access to the libraries available. Let’s talk through some of this and show you what it looks like.
00:22:08.520
First, regarding tooling: the JVM provides a wide range of garbage collectors. There are collectors that will run in parallel with your application, meaning you won’t see pauses. The garbage collection process utilizes all available cores to run as fast as possible.
00:22:32.040
Generally speaking, garbage collection should not be a problem for you on JRuby. If you’ve experienced issues on CRuby at all, there are lots of tools for monitoring the system—including the JIT and the garbage collector.
00:22:55.740
You can diagnose problems, pinpoint the source, and do all this remotely. You can connect to an existing application, even in production, and see what's happening. One of my favorites is a free tool that comes with the OpenJDK, called VisualVM.
00:23:19.130
You simply connect to your running application and monitor how everything's working. On the left side, you have a simple monitor of CPU usage and the heap's total size.
00:23:43.720
On the right is a plugin called Visual GC, which provides a live view of the actions of the garbage collector for your application—showing how generations fill up and how garbage is processed.
00:24:07.400
Every step of the way, you can monitor what’s happening in the garbage collector and see if you have memory leaks, or if you’re spending an excessive amount of time processing garbage.
00:24:27.650
This visibility is very useful, and you can connect this to a production application any time you want. It has profiling tools, which can be slightly mangled names for the Java VM, but they help you monitor how much time is being spent in various methods.
00:24:50.180
JRuby has its own optimizing interpreter that runs your code for a while, observes what’s happening, and determines what objects you’re using and what methods you’re calling. Our optimizing process follows.
00:25:09.720
If code turns out to be hot and looks like you’re using it a lot, we can convert it into JVM bytecode, and the JVM ultimately converts that into native code. JRuby was the first native JIT for Ruby, and it became available almost ten years ago.
00:25:32.840
With the introduction of Java 7, there’s a feature called invoke dynamic that significantly improves performance by optimizing dynamic calls, making them run like regular static Java calls.
00:25:57.420
We’re going beyond that to optimize Ruby code even further before it turns into JVM bytecode. We’re doing some inlining and optimization for numeric algorithms, translating them into raw Java primitives to enhance performance.
00:26:19.830
Groovy has also emerged as a new JIT for Java written entirely in Java. This allows us to optimize how Ruby works more effectively. We can direct the JVM on how to optimize Ruby better, leading to promising results.
00:26:40.950
For example, we have a benchmark that represents an array with blocks and loops, where we see how much faster JRuby is compared to CRuby at 2.4. Without invoking dynamic tricks, our basic JIT shows about a quarter times faster performance.
00:27:04.040
With invoke dynamic, it helps a bit, but since the JVM struggles to optimize blocks effectively, we still have room for improvement. However, we can further optimize the loop by inlining the code, eliminating the overhead of block dispatch.
00:27:27.390
This brings significant improvements—by the time we include invoke dynamic, we achieve performance that’s about eight times faster than CRuby 2.4! Growl helps us leverage optimizations, particularly for numeric algorithms.
00:27:52.200
One notable challenge JRuby faces is the need to use objects for numbers in the JVM, which means creating new objects each time math is done. However, if we optimize correctly, we can reduce that object overhead significantly.
00:28:16.890
For example, this Mandelbrot set runs significantly faster on JRuby, demonstrating nearly three times improved performance over Ruby 2.4. Pulling in Growl and optimizing numbers can lead to even better results across Ruby projects.
00:28:39.920
Now, let’s turn to concurrency and parallelism—what are these terms? They often get confused. Parallelism refers to the hardware aspect, where two pieces of code execute simultaneously, utilizing multiple cores.
00:29:01.740
In contrast, concurrency is a software concept where different tasks progress at the same time without necessarily executing at the same moment. You cannot have concurrency without parallelism. In contrast, CRuby allows multiple threads to progress, but at any one time, only one core will be active.
00:29:27.640
In JRuby, you can leverage true concurrency and parallel execution with a single JRuby instance. With larger Rails applications, having just one JRuby can saturate your system and use far less memory compared to multiple individual Rails instances.
00:29:53.860
Finally, let’s explore the fun areas with JRuby, particularly with Java libraries. You can seamlessly call Java code just like Ruby—just import the class or call it directly, and JRuby takes care of the rest.
00:30:13.780
Here’s a simple example using the Swing GUI library to create a frame, add a button, and set it visible. Most of this looks like regular Ruby code, even with Java's method calls.
00:30:36.300
We even have wrappers for JavaFX, allowing for declarative interface development through JRuby. We’ve worked closely with BreakMan Pro—a commercial tool for analyzing Ruby code using JRuby and JavaFX.
00:31:04.080
Tom also created a fun project called Prague, which enables writing Ruby plugins for Minecraft. There’s a bit of code that modifies the number of chickens that come out of an egg—setting it to 120. Tom quickly learned that it could lead to chaos.
00:31:27.140
So, what's next? First, JRuby is not a new project. We've existed for a long time, and many companies are using JRuby in real applications—this is just a small sampling of some organizations.
00:31:43.920
These include organizations like Visa and the BBC, which uses JRuby in their backend for election result services. If you're using JRuby, I’d love to hear from you and talk about what you’re utilizing it for.
00:32:03.980
Now, how can you get involved in JRuby? You can download the tarball. Unlike CRuby, there’s really nothing to build—just unpack it, put it in your path, and go! You can also run your favorite Ruby installers.
00:32:22.740
The only prerequisite is to get the OpenJDK installed, which is available for every platform. There are Linux packages that may lag a little, but they’re keeping up fairly well. For Windows developers, good news: JRuby operates almost the same on Windows as it does on UNIX.
00:32:46.760
Most of our extensions are Java-written, which means they work across platforms. We provide an installer to make JRuby easy to run on Windows as well. Additionally, if you’re tech-savvy, we welcome help with non-development tasks.
00:33:08.460
Take a look at our open issues; we currently have around 650 open ones. Many of these aren’t actual bugs, but rather issues that need updates. Going through the list, fixing what’s fixed or invalid will vastly help us.
00:33:32.570
You can also add to our wiki and improve documentation based on your experiences. If you're active with JRuby, feel free to communicate through our mailing list or social media. We’d love your input!
00:34:07.190
So if you're eager to contribute as a Ruby developer, a large part of JRuby is implemented in Ruby itself. If there’s a missing feature from MRI, you can code that in Ruby, and we’ll integrate it into the core.
00:34:24.420
For Java developers, we cherish your help to assist with internals and bolster JRuby. Our core classes are implemented largely in Java for performance and tooling.
00:34:41.720
We’ve kept the Java clean and readable. And if you're keen on compiler technology or interpreters, we’d love to have your assistance making improvements. Here are some links to help you get involved.
00:34:59.560
We have an IRC channel, and although we’re on GitHub, we try to keep our attention unified. Monitor both venues to connect with the JRuby community. Thank you for your time—try out JRuby!
Explore all talks recorded at EuRuKo 2017
+13