Summarized using AI

Building JRuby: How We Implement Ruby on the JVM

Charles Nutter and Thomas E Enebo • November 13, 2024 • Chicago, IL • Talk

The video, titled "Building JRuby: How We Implement Ruby on the JVM" presented by Charles Nutter and Thomas E Enebo at RubyConf 2024, explores the complexities and technicalities involved in implementing JRuby, the Ruby language runtime that runs on the Java Virtual Machine (JVM). The talk begins with an introduction to the JRuby project, highlighting its current developments and the necessity for commercial support following the loss of its major sponsor. Nutter emphasizes the advantages of JRuby, including access to Java libraries, improved scalability, and performance enhancements through the JVM's capabilities.

Key points discussed include:

- Current Status and Roadmap: The presenters provide an update on JRuby, noting its compatibility with Ruby 3.1, future plans for JRuby 10, and the transition to supporting Java 21. They also mention the deprecation of older Java versions.

- Building and Contributing: Instructions on setting up a JRuby environment are given, along with a brief overview of the project’s codebase and essential directories for contributors.

- Internals of JRuby: Technical discussions about JRuby signal processing, including information about its two parsers, the internal representation of code, compilation pipeline, and optimization through Just-In-Time (JIT) compilation.

- Core Classes and Compatibility: Highlighting the implementation of Ruby’s core classes such as String and Array, Nutter sheds light on how JRuby attempts to maintain compatibility with CRuby while leveraging Java features for optimization.

- Java Integration and Native Extensions: The speakers illustrate ways to integrate Java libraries seamlessly within Ruby, emphasizing that any JVM library can be used directly from Ruby code without additional compilation. They also discuss the conversion of native C extensions to a Java-based API to maintain efficiency.

- Community Involvement: The presenters encourage audience members to participate in the JRuby community by testing libraries, contributing bug fixes, and collaborating through communication platforms like Matrix.

In conclusion, they highlight the importance of community support in the ongoing development of JRuby and the promising advances being made through collaborations with projects aimed at improving JVM performance. The webinar ultimately paints an optimistic outlook for JRuby's future in the Ruby development ecosystem.

Building JRuby: How We Implement Ruby on the JVM
Charles Nutter and Thomas E Enebo • November 13, 2024 • Chicago, IL • Talk

What does it take to build a language runtime on the JVM? This talk will show how we implement JRuby, from Java versions of core classes and native extensions to our parser, compiler, and JIT subsystems. You'll learn how to find and fix bugs, profile Ruby code, and integrate with the many libraries and features of the JVM.

RubyConf 2024

00:00:15.559 yes all right I think we're good um I'll
00:00:19.359 get started you can take care of that
00:00:21.080 I'll fix this okay uh there we are all
00:00:25.880 right yes so J Ruby guys back together
00:00:28.320 finally uh 5 years since we presented
00:00:30.880 together at Ruby conf uh really glad
00:00:33.000 that we were able to get together here's
00:00:34.719 some information for us from us uh I'm
00:00:37.360 co-founder of headiest Enterprises which
00:00:39.040 I'll talk about in a second and Tom is
00:00:42.000 looking for work okay uh so I mentioned
00:00:47.480 uh hus Enterprises uh recently the jreu
00:00:50.399 project lost its major sponsor so we've
00:00:53.280 both been looking for ways to keep the
00:00:54.879 project going uh I have started a
00:00:56.920 company headiest Enterprises to do
00:00:59.239 commercial supp support for jruby users
00:01:01.719 uh you can grab this QR code if you or
00:01:04.000 anybody you know is interested in
00:01:06.600 migrating to jruby to get better
00:01:08.119 scalability or you've got a JB
00:01:10.040 application that you would like some
00:01:11.720 professional support for uh we've got a
00:01:13.799 wide range of options to help you with
00:01:15.479 that uh along those same lines we've got
00:01:17.840 stickers finally a new reprint of jruby
00:01:20.640 stickers uh that have all the
00:01:22.720 information about jruby support on them
00:01:25.200 please come up afterwards and and grab a
00:01:27.159 couple to to bring back to your your
00:01:29.119 teams
00:01:31.280 uh so a little intro on J Ruby we're
00:01:32.920 going to be talking mostly about J Ruby
00:01:34.439 internals here uh but of course J Ruby
00:01:36.720 is Ruby for the jvm and the flip side
00:01:40.280 bringing the jvm to Ruby all of the
00:01:42.360 capabilities of the Java platform with
00:01:45.280 good jit compilers amazing garbage
00:01:47.479 collectors uh parallel execution and
00:01:50.200 deploying across lots of systems we've
00:01:52.640 also got this giant ecosystem of java
00:01:55.280 libraries Java tooling uh really brings
00:01:58.479 a whole large world to Ruby and we also
00:02:02.280 get to use Ruby in the environment that
00:02:03.920 we're familiar with uh the development
00:02:06.240 experience is pretty much the same you
00:02:07.680 use the same command lines uh all the
00:02:10.000 same tools run we have a very high level
00:02:12.319 of compatibility uh give it a shot and
00:02:15.200 you know come by the hack fest tomorrow
00:02:16.760 we'll see if we can get some stuff
00:02:18.560 running okay so we're going to do a very
00:02:21.080 short road map update uh our current
00:02:23.879 release of J rubies J Ruby 94 it's
00:02:26.959 compatible with Ruby 31 so it's getting
00:02:29.160 a little long in the tooth for
00:02:31.200 compatibility uh we just put out our
00:02:33.080 last release last week and the big
00:02:35.760 statement is after we release our next
00:02:38.360 version jrb 10 we're going to give
00:02:40.560 approximately one year until we end of
00:02:42.280 life this but we've been pretty pretty
00:02:44.440 LAX on this if someone actually needs a
00:02:46.159 little longer we usually don't leave
00:02:47.879 people
00:02:48.959 hanging um the next big thing is J Ruby
00:02:51.760 10 it's going to update and catch up to
00:02:53.879 see Ruby and in fact up until the point
00:02:57.080 that we had lost our jobs uh we were
00:02:59.879 going to release this at the exact same
00:03:01.720 time as cruby but we've only we've only
00:03:04.840 drifted a couple of months and and we
00:03:07.159 should be able to make it and they're
00:03:09.319 out we wait till we come back do we get
00:03:12.959 any bonus time uh so uh the big change
00:03:17.480 is that we're going to stop supporting
00:03:18.920 Java 8 we're going to go up to the
00:03:20.959 newest long-term support version of java
00:03:23.120 which is Java 21 and Charlie's going to
00:03:26.599 oh wait no I'm still gonna go
00:03:30.560 um so the good news with 10 is all the
00:03:33.799 language features are completely
00:03:35.439 supported the only thing that we haven't
00:03:37.480 done is there's still some core apis
00:03:39.400 that need
00:03:40.480 updating uh so it shows like maybe 450
00:03:45.040 erors between the two test Suites but
00:03:47.560 there's a lot of duplication and like
00:03:49.920 one missing keyword might be 20 spec
00:03:52.040 failures so we're actually quite close
00:03:54.280 to being
00:03:56.159 done there's two of those for some
00:03:58.200 reason I'll do it a second time all
00:04:01.439 right so before we get into the the
00:04:03.760 internal details of J Ruby we'll do a
00:04:05.799 quick little walk through of the how to
00:04:08.200 get involved in the project uh so
00:04:10.959 working with J Ruby building it is very
00:04:12.720 simple the hardest part is probably just
00:04:14.840 installing a jdk use whatever package
00:04:17.199 manager you've got go and download one
00:04:19.120 of the many distributions that are out
00:04:20.519 there they'll all work just fine uh you
00:04:22.880 clone our repository you run the maven
00:04:26.199 command we've got built into the
00:04:27.840 repository itself and that's it
00:04:30.240 you have a completely working J Ruby
00:04:32.199 environment that you can do all of the
00:04:34.199 same Ruby stuff that you would with
00:04:35.440 regular CR Ruby except it's now going to
00:04:37.320 run on top of the
00:04:38.840 jvm uh quick codebase tour here most of
00:04:42.479 the core of J Ruby is in the core
00:04:44.360 directory uh there's mostly Java code
00:04:47.039 but also a little bit of Ruby for some
00:04:48.800 of the implementation the lib directory
00:04:51.240 is where all the gems and standard
00:04:52.759 libraries are uh the test and spec
00:04:57.160 directories are where we have all of our
00:04:59.000 various test Suites we run a couple
00:05:01.720 dozen different test Suites to confirm J
00:05:04.639 Ruby works
00:05:07.360 properly and uh and then the bin
00:05:09.360 directory is where all the gem
00:05:10.560 executables go and that's where the J
00:05:12.160 Ruby launcher
00:05:13.919 is uh I mentioned the kind of the ratio
00:05:16.479 of code in here uh it's actually has
00:05:18.800 gone down over the years it used to be
00:05:20.560 around 400,000 lines of java code now
00:05:23.400 down a little less than 300,000 lines
00:05:26.560 maybe change the C port and we're
00:05:28.720 pulling more code from uh from java into
00:05:32.639 Ruby as we go on it makes it a little
00:05:34.759 easier for us to optimize and it's a
00:05:36.520 little bit better for maintainers uh
00:05:38.880 contributors from the Ruby Community to
00:05:40.479 help
00:05:42.520 out try and continue
00:05:44.960 here so once you've got it up and going
00:05:47.440 and you've are able to build it and
00:05:48.680 you've got the code uh we recommend
00:05:50.800 intellig as the IDE that we use it's
00:05:53.560 what we use on a daily basis uh but
00:05:56.120 other editors and ID out there will work
00:05:58.120 just fine as long as they've got support
00:05:59.639 for a Java code base uh there's plenty
00:06:03.000 of jvm profiling and monitoring tools
00:06:05.080 out there I've done other talks on them
00:06:07.039 that you can look up we're not going to
00:06:08.240 go into some of that here but we can
00:06:10.680 benefit from all of the tooling on the
00:06:12.680 jbm without uh Without Really any
00:06:16.240 changes you can run your Ruby
00:06:17.680 application on the jvm and use the same
00:06:19.720 tools that the Java folks are used
00:06:22.160 to all right and I'll hand it back to
00:06:24.160 Tom here yeah so now we're going to talk
00:06:26.360 about some of the core pieces of our
00:06:28.919 codebase
00:06:30.160 we'll start with the
00:06:31.759 parser uh J10 actually has two parsers
00:06:35.319 the parser that we've been using for the
00:06:36.720 last 20 years and the new parser which
00:06:39.960 is prism it's an external open source
00:06:43.039 project and uh for the most part if you
00:06:45.919 want to contribute to this we can ignore
00:06:48.039 the parser part of this um but if you do
00:06:50.560 want to work on parsers uh go and
00:06:52.919 contribute to
00:06:54.840 prism the next big chunk is our internal
00:06:58.080 representation
00:06:59.919 uh this is largely changing the tree
00:07:02.400 that the parser generates into a
00:07:03.919 sequence of
00:07:05.000 instructions the instructions are pretty
00:07:07.160 obvious like we have a call instruction
00:07:09.000 for calling a method Define method I'll
00:07:11.680 let you guys figure that one out um but
00:07:14.560 when we need to implement something like
00:07:16.080 a a if statement or a loop we need some
00:07:18.720 branches and jumps so we add a few I
00:07:21.520 know computer sciency instructions and
00:07:24.879 uh the operands are also fairly obvious
00:07:27.479 string hash array local variable we have
00:07:30.800 a temporary variable that's new so at
00:07:33.199 the bottom here we're
00:07:34.960 calling uh 1 plus two and we're storing
00:07:38.639 it to a temporary
00:07:40.039 variable let's just go through a simple
00:07:43.080 example where we go and look at a single
00:07:45.639 node and we're going to build
00:07:46.800 instructions from it so if we look at IR
00:07:49.960 Builder the thing that builds these
00:07:52.280 instructions uh the first thing we have
00:07:54.159 to do is we have to get the global
00:07:55.639 variable using get Global variable for
00:07:57.440 dollar uncore we save it to a
00:08:00.520 amp and then uh we add a match
00:08:04.280 instruction that binds that dollcore
00:08:06.800 variable to the regular expression from
00:08:08.960 the
00:08:09.960 tree and so uh generally we only have to
00:08:14.120 change IR Builder when there's new
00:08:16.400 language features but sometimes we
00:08:18.520 realize we're doing something
00:08:19.840 inefficiently and we figure out a better
00:08:21.680 way of doing it so there's some
00:08:23.720 performance work but IR is much bigger
00:08:26.400 than just the Builder we have a bunch of
00:08:28.080 compiler passes we have two interpreters
00:08:31.400 and if you want to contribute to uh J
00:08:33.599 Ruby and you're interested in language
00:08:35.240 implementations this is a really fun
00:08:37.240 spot to
00:08:39.719 be all right so I'll talk a little bit
00:08:42.000 about where the IR goes from there so
00:08:44.600 here is the compiler pipeline of of J
00:08:46.760 Ruby we've got our Ruby code coming in
00:08:48.720 on the left uh that gets parsed and
00:08:51.040 compiled internally to our intermediate
00:08:53.120 representation our instructions very
00:08:55.240 similar to the yarv instructions on the
00:08:57.800 Ruby VM the C Ruby VM
00:09:00.200 uh and then we interpret those for a
00:09:01.440 while and that's kind of as far as Ruby
00:09:04.000 has gone uh since 1.9 create a list of
00:09:07.279 instructions run the interpreter but of
00:09:09.680 course in C Ruby and in J Ruby we want
00:09:12.760 to go beyond that we want to get rid of
00:09:14.839 The Interpreter overhead so our next
00:09:17.399 step then is to just take the Ruby
00:09:19.760 intermediate representation and compile
00:09:21.760 it into jvm bite code AS efficiently as
00:09:24.200 we can from there we don't have to do
00:09:27.320 any more work we let the jvm take it the
00:09:30.200 jvm will interpret it and profile it
00:09:32.720 pass it off to its jit compilers and
00:09:35.079 then over time iterate over it and get
00:09:37.000 the best performance out of it all we
00:09:38.800 had to do was get the jvm bite code in
00:09:40.839 there for this Ruby code and we get the
00:09:42.880 benefit of all of those
00:09:45.079 optimizations so I'm going to walk
00:09:46.760 through a little bit of the code that's
00:09:48.200 involved in executing ir and then how it
00:09:50.760 feeds into the jit compiler for J Ruby
00:09:53.959 so every one of the literal types float
00:09:56.680 fix num string Etc has an operand type
00:09:59.839 in our IR uh this is the float operand
00:10:02.399 part of the
00:10:03.399 implementation uh we can see it up at
00:10:05.839 the top here it extends the immutable
00:10:08.120 literal does that show up on there yes
00:10:10.839 there we go uh a mutable literal because
00:10:12.920 this is an object that we will create
00:10:14.760 once and then cache uh we have a final
00:10:19.079 double value here not too surprising a
00:10:21.320 simple Constructor and then once we get
00:10:24.120 to The Interpreter The Interpreter will
00:10:26.839 call into this use this operand to
00:10:29.320 determine how to create a float object
00:10:31.320 and then cache it for us so that we
00:10:32.920 don't use keep creating the same object
00:10:35.920 uh we also have this visit method which
00:10:38.200 is used for us to translate ir into
00:10:40.760 other forms uh we have a pretty printing
00:10:43.880 dumper that we'll print it out on the
00:10:45.519 console with color and and formatting so
00:10:48.519 you can see your IR uh but for the
00:10:51.000 purposes of this talk we go into the jvm
00:10:54.240 visitor which turns IR into jvm by code
00:10:58.079 uh so IR visitor is the super class here
00:11:00.440 is just a snippet a few of the different
00:11:02.160 operands that it you can Implement uh
00:11:05.200 and then eventually we end up in the jvm
00:11:07.160 version of that visitor so we feed in
00:11:10.000 our float operator or our float operand
00:11:13.880 uh we grab the current jvm method that
00:11:16.120 we're emitting we get the compiler that
00:11:18.720 emits values for it and tell it to push
00:11:21.040 a float onto the stack uh and most of
00:11:23.760 jvm visitor looks like this calls
00:11:25.680 through these abstract interfaces
00:11:27.560 because we have a few different ways to
00:11:28.920 compile things so for pushing a float in
00:11:32.360 our value compiler uh there are two
00:11:34.959 different ways that we might emit that
00:11:36.440 code one is sort of normal jvm bite code
00:11:40.519 uh we're going to Cache this value so we
00:11:42.760 have a helper to Cache that value
00:11:44.480 permanently we push our jruby runtime
00:11:47.440 onto the stack that's this code here uh
00:11:51.079 LDC is pushing a constant so we push the
00:11:54.079 constant double value onto the jvm stack
00:11:56.760 and then we call new float to create the
00:11:58.639 object and then this cache permanently
00:12:01.200 will save it for us and emit the right
00:12:03.279 jvm bite code to do it uh we also use a
00:12:06.160 feature on the jvm called invoke Dynamic
00:12:08.639 it allows us to teach the jvm how to
00:12:10.760 optimize Ruby code like it would Java uh
00:12:14.079 without having to do any native code and
00:12:16.639 down at the bottom is much simpler here
00:12:18.279 we have Push we uh have our double value
00:12:21.440 we load our current execution context
00:12:23.800 and then we just invoke Dynamic the
00:12:25.480 float operation and it passes in that
00:12:27.920 double uh I mentioned invoke Dynamic
00:12:30.800 here so this is basically a jvm bite
00:12:33.040 code for doing any Dynamic binding on
00:12:35.160 the jvm instead of just doing one of the
00:12:38.120 existing jvm operations we give it a
00:12:41.480 special object that says here's how you
00:12:44.120 optimize it here's how you turn this
00:12:46.040 into fast code uh so this has allowed us
00:12:49.279 to teach the jvm how Ruby can optimize
00:12:52.560 and get performance much closer to
00:12:54.199 Native Java and and eventually native C
00:12:57.680 code
00:12:59.880 uh so here is the bite code we get out
00:13:01.399 of this the bite code on top is the
00:13:03.920 normal-ish jvm bite code uh notice a get
00:13:07.399 static and put static at the beginning
00:13:09.320 and end that's basically checking if
00:13:11.240 we've created the object already if not
00:13:13.680 we store it in a static location so that
00:13:15.800 we don't have to recreate it every time
00:13:18.639 uh the invoke Dynamic version is only
00:13:20.480 two operations this is the loading of
00:13:23.440 our current execution context and then
00:13:25.760 our invoked Dynamic and the magic of
00:13:28.800 that so up here we have our new float
00:13:30.920 Creation The Magic of the invoke Dynamic
00:13:33.480 is we tell the jvm go to this class to
00:13:36.680 get the way of compiling this
00:13:38.880 instruction if we look at our float
00:13:41.079 object site this is what goes to the jvm
00:13:44.079 and says how to optimize a float
00:13:46.320 construction uh again we've got a it's a
00:13:49.600 lazy object we're going to create we
00:13:51.480 have our final double value in a simple
00:13:53.560 Constructor and then we construct the
00:13:55.600 new float pass it back to the jvm and it
00:13:58.040 caches it like a con
00:13:59.759 and interestingly this is almost
00:14:01.800 identical to the original operand that
00:14:04.040 we had in IR so we're basically making
00:14:07.360 new instructions for the jvm that are
00:14:10.160 simply a version of the The Interpreter
00:14:13.120 instructions but doing it in such a way
00:14:15.240 that the jvm can optimize it like Java
00:14:18.800 code all right so a little review before
00:14:21.040 I move on here parser creates the ab
00:14:23.399 abstract syntax tree uh the IR compiler
00:14:26.240 turns the into our instructions we
00:14:29.120 interpret those instructions for a while
00:14:30.680 and learn a bit about them eventually we
00:14:33.040 take our jit compiler and turn the
00:14:35.320 interpreted instructions into jvm bite
00:14:37.680 code and then the jvm takes off with it
00:14:40.399 and continues to optimize it uh but what
00:14:43.040 about all the other classes that we use
00:14:44.480 string and array and so on so let's go
00:14:46.839 into how J Ruby implements the core
00:14:48.600 classes of
00:14:49.959 Ruby uh so all the core classes like
00:14:52.839 string array hash and the numerics and
00:14:55.000 such are generally in uh or the org J
00:14:57.880 Ruby package Ruby string Ruby array Ruby
00:15:00.759 hash pretty easy to locate these uh
00:15:03.600 compatibility in J Ruby very closely
00:15:06.000 follows cr- Ruby so you'll notice if you
00:15:08.800 look at the code the implementation of
00:15:11.199 our core class methods and the
00:15:13.040 implementation of C Ruby are very
00:15:14.800 similar there's only so many ways that
00:15:17.079 you can concatenate two strings or add
00:15:19.519 two numbers so the the the logic is
00:15:22.040 often a line by line Port of the C code
00:15:24.360 from
00:15:25.279 cruby uh things that are different we
00:15:27.959 use Java featur that are available we'll
00:15:30.040 use more static types we can represent a
00:15:32.880 multiple aity method like a method that
00:15:34.959 takes one two or three arguments with
00:15:37.279 different entry points that helps us
00:15:39.440 optimize it avoids boxing things in an
00:15:41.639 argument array uh and it fits better
00:15:43.680 into the jvm so I'll walk through an
00:15:46.279 implementation here of the uh array
00:15:50.199 bracket operator the array reference
00:15:52.040 operator in the C code and then show you
00:15:54.160 what it looks like in J Ruby it's very
00:15:55.959 similar so if you look at array.c in MRI
00:15:59.560 in C Ruby uh you'll see these defined
00:16:02.120 method calls RB defined method uh and
00:16:04.639 this takes a class that we're going to
00:16:06.440 put the method on a name to assign to it
00:16:09.160 and then a pointer to a function
00:16:10.680 somewhere uh that represents the body of
00:16:13.120 that that that
00:16:14.920 method uh then we have RB AR RB array a
00:16:19.240 reference here uh again this is a
00:16:22.000 multiple aity a variable aity method uh
00:16:25.000 so it takes in an argument array and a
00:16:27.199 count of arguments and then has to the
00:16:29.040 branch based on how many arguments there
00:16:30.880 are we'll follow the one argument path
00:16:34.000 here so here is RB ar1 receives one
00:16:38.959 argument uh the bulk of the logic here
00:16:42.600 you'll see in the middle the switch and
00:16:45.279 this arithmetic sequence begin length
00:16:48.000 step this is for all the cases where
00:16:50.759 it's not a simple number it's either an
00:16:53.160 arithmetic sequence or it's a range or
00:16:55.480 it's some other structure we'll pass
00:16:57.839 that off to another meth method uh but
00:17:00.000 what we're really interested in is a
00:17:01.240 simple case here if it's a fix num
00:17:03.560 argument a numeric argument we go into
00:17:05.720 rbr area entry to actually grab the
00:17:07.880 array
00:17:08.799 entry RB area entry simple enough just
00:17:11.839 calls into an internal version of that
00:17:14.959 operation here's the internal version We
00:17:18.000 grab the length of the array we grab a
00:17:20.360 pointer to all the array contents do
00:17:22.880 some bounds checkings on it and then
00:17:25.400 down at the bottom actually access the
00:17:27.880 spot in memory the point in memory where
00:17:30.000 the value is contained so that's the C
00:17:33.039 implementation in J Ruby so I mentioned
00:17:36.120 that we can do uh specific aity versions
00:17:40.120 so this is the method implementation for
00:17:42.679 one argument we don't need to check the
00:17:44.799 argument count we don't need to uh raise
00:17:48.600 an error because we know if we get to
00:17:50.400 here we've already got one argument
00:17:52.000 there's no argument link checking you
00:17:54.440 can see both of the names that we assign
00:17:56.360 are up here in our jruby method
00:17:58.080 annotation so rather than having to
00:18:00.280 enter in multiple bindings of it we just
00:18:03.000 put more names in here uh and then
00:18:05.880 eventually we determine that it's a
00:18:07.320 fixed num and we call into our entry
00:18:09.760 method the entry method takes that value
00:18:13.120 does some bounce checking on its own
00:18:15.240 eventually bottoms out in this El in
00:18:17.600 timate element internal function and
00:18:20.760 that's where we actually access our
00:18:22.440 array internally in the jvm and then
00:18:24.880 give you the value back we also have
00:18:27.520 some specialized versions of these
00:18:29.000 classes for for more optimized cases
00:18:31.720 Ruby array one object as if it's an
00:18:33.640 array that only has one value in it uh
00:18:36.360 and the main difference here is that
00:18:37.960 there's no there's not a second level of
00:18:39.840 array in there there's just a field we
00:18:42.039 grab it makes a more compact object so
00:18:44.799 when there's a small number of elements
00:18:46.360 in an array we'll try to optimize it
00:18:48.240 down to one of these small sizes that's
00:18:50.080 specialized for it uh so how do you hunt
00:18:53.039 down and find these core methods uh most
00:18:55.840 of these classes are going to be in the
00:18:56.960 OR J Ruby package all of the methods
00:18:59.720 that we bind to Ruby have this J Ruby
00:19:01.559 method annotation so if you search for
00:19:03.520 that you can find them we also try and
00:19:05.960 include in our code some comments that
00:19:08.200 say what the equivalent cruby version is
00:19:10.919 so RB new if you search for it it'll
00:19:13.600 take you right to our new array
00:19:15.200 functions in J
00:19:16.720 Ruby and we'll talk a little bit more
00:19:18.559 about that API uh in the last section
00:19:21.280 here uh this is kind of a good place
00:19:24.559 where we can jump off into how we do
00:19:26.400 Native extensions uh so of course we
00:19:28.480 want use pure Ruby as much as possible
00:19:31.039 but sometimes we need more performance
00:19:32.919 or we need to call out to an external
00:19:34.520 library in C Ruby this is done with a c
00:19:37.240 extension on J Ruby we don't support the
00:19:39.520 same C extension API but we have
00:19:41.559 essentially an equivalent API for J Ruby
00:19:44.320 a Java extension API now when you're you
00:19:47.200 need an extension there's three things
00:19:49.080 that we would recommend you consider
00:19:50.799 first of all if you can just use pure
00:19:52.760 Ruby that's usually fine we optimize
00:19:55.720 very well often times you don't need to
00:19:58.320 got drop down into Java code uh to get
00:20:01.000 good performance out of that code on J
00:20:02.799 Ruby uh but maybe maybe it's a library
00:20:05.400 you need to call well if it's a jvm
00:20:07.600 library you can just call it from Ruby
00:20:09.720 code you can import the classes call all
00:20:12.080 the methods it looks like you're just
00:20:13.559 calling into another Ruby class uh
00:20:15.880 similarly with ffi you can pull in
00:20:17.919 Native libraries use the foreign
00:20:19.760 function interface and call all those
00:20:21.400 from Ruby code uh but if you really have
00:20:23.799 to you can Port the extension into our
00:20:25.880 jruby
00:20:26.960 API so quick couple quick notes on Java
00:20:29.919 integration uh like I say any library on
00:20:32.360 the jbm written in Java cotland Scola
00:20:35.320 closure whatever can be pulled into a
00:20:37.480 ruby application and called from Ruby
00:20:40.159 code without writing anything without
00:20:41.840 compiling any Java code or or jvm bite
00:20:44.520 code uh and we do a lot internally to
00:20:46.960 optimize this to make it work well we
00:20:49.840 also provide a bit of ecosystem help to
00:20:52.200 uh pull in library pull in dependencies
00:20:54.159 for libraries so that they can be
00:20:56.159 bundled into a gem uh the steps for
00:20:58.960 calling jvm libraries you require in the
00:21:02.240 jar file just like you would any other
00:21:04.039 Ruby Library Java import a specific
00:21:06.880 class into your local
00:21:08.760 namespace create the object call the
00:21:11.080 functions we provide both camel case and
00:21:13.159 underscore case versions of everything
00:21:15.559 uh and some niceties like turning Java
00:21:17.400 Getters and Setters into
00:21:19.840 attributes uh lots more examples on the
00:21:21.960 wiki I won't go into here is a quick
00:21:24.039 little walkthrough example so this is
00:21:26.200 bringing up a j Ruby IRB session
00:21:29.440 uh we are going to use the Java Swing
00:21:32.480 guey components here first of all we
00:21:34.799 created a jframe with the words J Ruby
00:21:37.480 is
00:21:38.159 awesome uh we create a button with the
00:21:42.120 it's going to tell you to push me press
00:21:45.520 me we add the button to the
00:21:48.520 frame we set the size of the frame 300
00:21:51.480 by 300 and then we show the frame there
00:21:55.000 we go we have our guey already done in
00:21:57.480 an interactive session
00:21:59.279 now we can bind some action listeners to
00:22:01.440 it so whenever the button is pressed we
00:22:03.520 want it to say you pressed me you push
00:22:05.559 the button and it calls the Ruby code
00:22:07.159 and executes and it's really that easy
00:22:09.799 to call into jvm libraries uh definitely
00:22:13.360 check out glimmer there's a workshop
00:22:15.080 tomorrow which can let you use a lot of
00:22:17.320 the same gooey libraries on J Ruby
00:22:19.440 without using any of the direct Java
00:22:21.039 calls
00:22:22.279 too
00:22:24.720 yes so I'm going to do a quick walk
00:22:26.919 through the psych parser as as an
00:22:28.760 example of a c extension that's
00:22:30.159 converted into Java uh the psych parser
00:22:33.679 has this while loop that basically
00:22:35.880 iterates over all the parser events it
00:22:38.200 feeds the yaml into lib yaml and pulls
00:22:41.200 off events until they're done so here we
00:22:43.880 check if there's any errors during the
00:22:45.360 parse we get our position information
00:22:47.799 and then eventually drop down into this
00:22:49.520 switch uh this example is for if there's
00:22:51.960 a yaml Alias uh we check if there's an
00:22:54.559 anchor create a string for it and then
00:22:57.039 RB protect down the bottom ends up
00:22:59.240 calling back out to Ruby feeding those
00:23:01.279 values out ours is very similar again
00:23:04.440 this was almost a line by line Port
00:23:06.480 we're using snake yaml instead of lib
00:23:08.440 yaml we get the next parser event we get
00:23:11.640 our position information then we have
00:23:13.919 our Alias that checks if there is an
00:23:15.720 anchor turns it into a string and again
00:23:18.240 does our call out to Ruby so it's very
00:23:20.799 similar C extensions can be ported very
00:23:23.120 easily uh and we're trying to make that
00:23:26.240 easier yeah however
00:23:29.960 we have a problem every Java method in
00:23:33.120 our codebase that's marked with public
00:23:35.880 is basically our API right now and we
00:23:39.480 have to use public to call across
00:23:41.120 packages internally so we're definitely
00:23:43.159 don't want to expose every method we
00:23:46.559 have and
00:23:48.520 so in order to prevent breaking native
00:23:51.760 extensions we basically will create a
00:23:54.120 new method and the old method will just
00:23:56.039 get deprecated so we have over 1500
00:23:59.120 deprecated methods in our in our core
00:24:02.000 right now and we need to fix those So
00:24:05.520 the plan is we're defining a new API uh
00:24:08.559 package space um so that known native
00:24:11.720 extensions can go and update to it like
00:24:15.080 psych and uh this API is going to be
00:24:18.039 back ported to 94 and we're actively
00:24:20.159 using it in 10 so now once we get an
00:24:23.760 extension to update to it they just
00:24:25.360 version lock to the latest version of 94
00:24:28.720 and then
00:24:30.440 they're they're no longer our
00:24:33.440 problem so right now we have three
00:24:35.919 classes with a bunch of static methods
00:24:38.600 uh we have create which allows you to do
00:24:40.240 things like new array new symbol new
00:24:42.600 string uh air which allows you to go and
00:24:45.200 create the right uh Ruby exceptions that
00:24:47.919 you want to throw but it also has a nice
00:24:49.880 formatting that you expect uh convert
00:24:53.000 which allows you to go and say I have a
00:24:54.480 ruby object but it needs to be a ruby
00:24:57.000 range uh if if you have a Java long you
00:24:59.279 can convert it into a a ruby fix num and
00:25:03.000 here's just a quick example um there's
00:25:06.480 two of these too oh oh you did this
00:25:09.480 different okay never mind um so there's
00:25:11.799 three methods that we're using we're
00:25:13.880 actually dog futing this internally in
00:25:15.360 our own code base so the first thing
00:25:18.200 that we um call is cass's range this
00:25:21.919 makes sure that it's actually a ruby
00:25:23.679 range and if not it does a type
00:25:25.799 error uh here's an argument error which
00:25:28.720 is a nice nicer syntax than what we had
00:25:31.799 before and then lastly we need to get
00:25:35.000 the begin and end for this clamp
00:25:37.159 implementation and compare so if we go
00:25:39.679 over and look at uh Ruby range you can
00:25:42.360 see that we have these annotated as
00:25:43.840 being a bless API that we can
00:25:46.159 use so uh because we have these uh The
00:25:50.679 annotation in this API space will'll
00:25:52.480 generate a nice guide we're going to
00:25:54.559 have an annotation that's going to allow
00:25:56.559 people who are used in doing native
00:25:58.520 extensions to be able to say well what
00:26:00.760 Java method should I call and lastly if
00:26:03.720 you haven't contributed to J Ruby and
00:26:05.399 you'd like to you can uh get involved in
00:26:09.120 this this is going to be the easiest way
00:26:10.679 to get involved it's just translating
00:26:12.559 one method to
00:26:14.320 another all right so closing out here
00:26:17.080 how can you help and get involved in J
00:26:19.360 Ruby um we definitely need your help uh
00:26:22.600 working on J Ruby itself there's
00:26:24.480 complexity and there's challenges there
00:26:26.520 but the biggest way that you could help
00:26:28.080 is just try out your applications try
00:26:30.440 out libraries if you've got a library
00:26:32.399 that you really depend on or that you
00:26:33.799 like make sure that the tests run in J
00:26:36.120 ruy make sure they're running in CI um
00:26:38.679 if you find issues file bugs anything
00:26:41.279 that anything that comes up just file it
00:26:43.120 maybe it's not our problem we'll work
00:26:45.000 through it with you and figure it out uh
00:26:47.120 maybe you can help contribute by looking
00:26:48.880 at the existing bugs and fixing them we
00:26:51.080 can write it in Ruby if you want to
00:26:52.679 we're fine taking in Ruby code doesn't
00:26:54.919 have to be Java we'll we'll optimize if
00:26:56.799 necessary later um I mentioned adding J
00:27:00.159 J Ruby to popular libraries make sure
00:27:02.320 that they have J Ruby support make sure
00:27:04.120 they're running us inci uh these are the
00:27:06.320 things that we really need to stay fresh
00:27:08.360 and and part of the
00:27:10.279 ecosystem uh join the community we have
00:27:12.960 a matrix channel that we're on all the
00:27:14.840 time uh Matrix is an open source open
00:27:17.799 platform for doing slack like
00:27:20.039 collaboration that's where we've put our
00:27:21.640 J Ruby Channel there is also a Google
00:27:24.279 group mailing list mostly just gets jby
00:27:26.720 announcements but there are occasional
00:27:28.200 conversations there uh you can also
00:27:30.200 contact Tom and I directly this is just
00:27:32.200 what we do all the time so please reach
00:27:34.600 out to us uh and there is some of our
00:27:38.039 contact information and websites we are
00:27:40.320 going to be at the hackfest tomorrow uh
00:27:42.480 so if you've got a library or
00:27:44.159 application you want to try out please
00:27:46.320 stop by or if you'd like to get involved
00:27:48.679 and help work on J Ruby do that too so
00:27:51.440 hopefully we'll see you tomorrow and I'd
00:27:53.519 like to point out one other thing this
00:27:55.600 guy's looking for a job so
00:27:59.200 yes hire him hire him to work on J Ruby
00:28:01.600 that would be great yeah all right thank
00:28:09.840 you have time for one or two
00:28:14.480 questions yeah yes you are you
00:28:20.440 excited yes ah somebody who's been
00:28:22.880 following this so Liden is a jbm open
00:28:26.320 jdk project to uh reduce startup time
00:28:29.559 essentially we're really excited because
00:28:31.600 it's probably our the biggest complaint
00:28:33.159 we get about J Ruby is startup time is
00:28:35.279 just a little too slow uh I've done some
00:28:37.679 early experiments with it it's looking
00:28:39.480 very promising and I'm working directly
00:28:41.480 with the lien folks uh to help that out
00:28:44.559 uh Valhalla is bringing some features
00:28:47.240 like value type stack allocation to the
00:28:49.360 jbm we will definitely take advantage of
00:28:51.720 those to reduce some of our numeric
00:28:53.679 object overhead um I remove some of the
00:28:56.240 slides here for time but we're also
00:28:57.720 interested in Project Panama which will
00:28:59.720 make ffi Native C calls very fast
00:29:02.519 already starting to use that and project
00:29:04.480 Loom which makes our our Ruby fibers
00:29:07.880 scale much higher tens of thousands
00:29:09.880 hundreds of thousands of fibers we're
00:29:11.559 already using that and you already get
00:29:12.960 the advantage of you if you pull down a
00:29:14.760 new jvm so we're tracking all of those
00:29:16.720 open jdk projects they all can
00:29:19.360 help well well thanks for coming today
00:29:22.320 yes thank
00:29:27.039 you for
Explore all talks recorded at RubyConf 2024
+64