One cable to rule them all

00:00:01.190 Okay, let's start. Hi, my name is Vladimir, and today I'm going to talk about "One Cable to Rule Them All".

00:00:06.930 Actually, the name of this cable is AnyCable. Let me introduce myself.

00:00:12.900 I’m Vladimir, and I traveled to Matsushima two days ago. It’s a pretty cool city, but I came from far away—Moscow, Russia. Even in Russia, Moscow is quite far from Japan; it’s a 9-hour flight. You can find my GitHub and Twitter profiles with these handles, feel free to check them out.

00:00:30.570 Now I'm working at a company called Evil Martians, and actually, today is my second anniversary at Martians.

00:00:38.100 What is Evil Martians? We do product development for big companies and small companies all over the world, mostly in the United States. Of course, we’re also involved in a lot of open source projects, focusing on front-end technologies, Ruby, and various online tools.

00:01:05.820 We write about these topics in our beautiful blog, and you might have read our articles translated into Japanese—thanks to Hachi for that. Today, I'm going to talk about cables and a couple of particular cables.

00:01:20.790 Let’s start with the first part: What is a cable? By 'cable,' I mean any tool used to develop a real-time application—libraries, frameworks, services, and so on.

00:01:34.380 Real-time applications are those that require some form of synchronous communication between the client and the server. Think of messaging, live notifications, online games, and so forth. In the Ruby world, we do have dozens of different cables and libraries available to build real-time applications.

00:02:07.000 Unfortunately, we often find ourselves relying on non-Ruby cables because there are languages and technologies that handle real-time tasks and support concurrency out of the box. As we've learned today from Chichi’s talk, Ruby still has limitations due to the Global Interpreter Lock (GIL), which may not provide the concurrency needed for high-load situations.

00:02:39.910 So, what should we do? Does that mean we should leave our wonderful Ruby world for something else? My answer is no; we shouldn't. In this talk, I'd like to show you how we can write high-performance real-time applications using Ruby as the primary language.

00:03:04.420 Let’s start with a problem called Action Cable. What is Action Cable, and why have I labeled it a problem? Action Cable is currently the most popular cable because it’s integrated with Rails, which is the most popular framework.

00:03:38.350 First, let me ask: Who is familiar with Action Cable? Raise your hands. Okay, cool! And who is using it in production? That’s great!

00:04:01.900 Action Cable consists of a server that handles connections, a broadcaster responsible for subscribing clients to streams and transmitting messages, and an abstract layer called channels, which serves as the framework to describe the logic behind real-time communication. The channel class itself looks very similar to a controller class, acting as a kind of controller for your WebSockets.

00:04:50.460 The cool thing about Action Cable is that it allows you to write and work with real-time applications very quickly—almost in five minutes, which is quite appealing. However, we need to ask whether it really works effectively in a production environment.

00:05:25.700 Does it scale? Can it handle a significant load? These are important questions.

00:05:47.520 Let's consider some benchmarks to illustrate the performance issues with Action Cable. One benchmark I’d like to reference compares different WebSocket applications written in various languages, including Erlang, Elixir, Closure, and of course, Action Cable.

00:06:10.030 The benchmark measures the time it takes for the server to broadcast a message to all clients. To illustrate, imagine I’m the server, and you are all the clients. If someone says, "Ruby is cool!" my task is to broadcast that message to everyone.

00:06:47.920 We then measure the time taken to transmit that message, as it is a critical performance metric for real-time applications.

00:07:05.380 The faster we can broadcast messages, the better our real-time application performs since nobody wants to wait a long time for a chat message to appear.

00:07:32.020 Looking at the benchmarks for Action Cable with different numbers of processes, we see that when it comes to dealing with thousands of connections, Action Cable does not perform as well.

00:07:55.930 Although it is still usable, we have to be cautious—having streams with thousands of subscribers is problematic. If your streams have dozens to hundreds of subscribers, things may look good, generally with latency being around one second. However, once a stream becomes crowded, the latency increases and the definition of 'real-time' diminishes.

00:08:29.020 Furthermore, no matter what your broadcasting method is, resource usage is very high. Let’s take a look at some graphics illustrating CPU usage when broadcasting to thousands of connections. As seen, the CPU does not handle this well.

00:09:19.670 The chart also highlights memory requirements for handling 20,000 idle connections. These connections aren’t doing anything; they are simply connected, and the amount of memory needed to maintain such connections is considerable.

00:09:51.420 Benchmarking is not the perfect method for learning, so let me share an anecdote regarding real-time data.

00:10:12.060 The project I worked on is called EquipComm, which doesn't matter much, as it's complicated. I previously used Action Cable in production on Heroku and required modern 21x dynos to handle about 5,000 connections, which took up 20 gigabytes of RAM due to memory issues.

00:10:49.420 You may wonder why so much memory is required to manage real-time functionality with Action Cable. One reason is that Ruby’s WebSockets are generally implemented using an architecture known as Rack Hijack, which allows you to directly access underlying IO, namely TCP sockets, and manage the process manually within Ruby.

00:11:33.860 We need a separate IO loop to handle concurrent connections and must parse the WebSockets protocol manually. While Ruby does have C extensions for better performance, they aren’t widely adopted. There is also a new Rack API proposal intended to help extract low-level work to web servers, potentially involving C extensions.

00:12:00.950 There is another problem regarding Action Cable and other implementations. They follow a Ruby-like, object-oriented approach and maintain a persistent state. Each Action Cable connection necessitates the creation of several unique objects across different abstraction layers just for initialization.

00:12:37.590 This accumulation results in high memory usage. Additionally, Action Cable has a problematic pub/sub implementation, as it performs JSON encoding for every client during each broadcast, which introduces a computational overhead.

00:13:05.170 This overhead affects memory allocation since strings are heavy, particularly when encoding JSON. I even proposed a pull request to Action Cable to ameliorate this inefficiency.

00:13:43.130 These issues can lead to heap fragmentation. I can illustrate this with a heap dump of an Action Cable process after some stress testing, which shows considerable empty spaces.

00:14:10.720 Calculating the outcome reveals that about 60% of the heap is empty, but due to fragmentation, allocation isn’t possible. This fragmentation partially explains why, in production, more memory must be added when using Action Cable.

00:14:45.250 So, what could be the possible solutions? One option is to wait for the GIL to be resolved, implement a new Rack API or migrate to languages like Elixir, Closure, Go—though these aren’t necessarily interesting for the Ruby conference.

00:15:00.509 Let’s explore another solution called AnyCable. What is AnyCable? It’s designed to improve and address issues found within Action Cable by combining its logic with other technologies like Go.

00:15:34.190 AnyCable comprises four layers, with the layer responsible for low-level message sending and connection management being the server. Imagine if we could migrate this server outside of our Ruby processes, which is precisely what AnyCable does.

00:16:04.970 It keeps the Ruby parts of Action Cable—the channels, and your existing code—while handling all the low-level tasks related to sockets and broadcasting.

00:16:41.000 In this setup, WebSocket clients connect to the WebSocket server, which communicates with the Ruby on Rails application to perform necessary actions such as subscribing or executing RPC calls.

00:17:12.80 The question remains: How can we efficiently communicate between a WebSocket server, which uses a different programming language such as Go, and Ruby, particularly when building high-load applications?

00:17:50.90 The answer is gRPC, a universal RPC framework developed by Google. gRPC lets us write our servers and clients in different languages and facilitates easy communication.

00:18:15.930 Built upon HTTP/2 and Protocol Buffers, gRPC allows us to utilize streaming capabilities for message passing alongside descriptor files to define our services.

00:19:06.230 The AnyCable service is quite simple, only containing three public methods to invoke. When a client connects, we authenticate the connection, and for any other action, we call the corresponding command. Finally, when the client disconnects, we invoke the disconnect method.

00:19:44.590 The final diagram looks like this: we have a Go application communicating with the RPC server over HTTP/2. For broadcasting within other parts of our application, we temporarily use Redis.

00:20:08.450 One question that may arise is: How good is this connection? Could it present a bottleneck due to HTTP overhead? It turns out that performance remains solid.

00:20:51.490 In fact, we achieve reasonable figures based on the concurrency level. Although it’s subject to Ruby’s implementation limitations, it's sufficient for building a solution atop it.

00:21:27.030 Back to the story: after using AnyCable for the project EquipComm, we reduced the number of dynos by a significant margin, which is impressive as it saves costs.

00:22:04.750 What are the key features of AnyCable? It doesn’t maintain long-lived objects in your Ruby process; instead, it encodes and stores the state within the WebSocket server and lazily restores it when needed.

00:22:35.780 The implementation remains straightforward. AnyCable acts like a plug-and-play solution—adding the gem and running the WebSocket server gives you a high-performance outcome.

00:23:21.210 In terms of compatibility, while some features may not be supported yet, the majority are. If you're looking to improve Action Cable performance, I highly recommend giving Whatever AnyCable a shot!

00:23:54.500 Also, AnyCable isn’t exclusively a Rails project; it can operate without Rails. However, you will still need to have some code written, which brings us to LightCable.

00:24:15.790 LightCable is a companion project that implements the Action Cable channels framework with no dependencies. It’s compatible with AnyCable and Action Cable clients. It is designed to be used with pure Rack applications.

00:24:46.359 This brings us to the final chapter that I find the most interesting. How can we further enhance AnyCable? Yes, we are striving for perfect performance.

00:25:01.830 Most channel implementations are straightforward, and often they do not contain heavy specific logic. You can simply subscribe a client to a stream without needing to call RPC.

00:25:32.720 If we could eliminate the unnecessary RPC calls, this would be beneficial. So, what options exist? Have you heard about the jRuby language? It is a Ruby-like project but written in Go. I was considering using it because it could be compatible with our needs.

00:26:02.090 Unfortunately, it turns out the integration isn’t possible since you can’t use it as a Go library.

00:26:30.460 As an alternative, there's a project called M Ruby. It's an embeddable Ruby implementation used in different contexts, and people are relatively familiar with it.

00:27:01.540 I’ve experimented with it and created a command line AnyCable client across platforms. In fact, I previously contributed to Ruby itself.

00:27:36.370 Now, here’s a quick note: for better readability, I've removed all error handling from my Go examples because they could clutter things up.

00:28:05.200 So, RubyGo is a wrapper around the M Ruby VM, allowing it to interface with Ruby. You need to initialize a context that simulates the behavior of a Ruby VM.

00:28:47.120 Once initialized, we can load Ruby code and perform it as needed. The implementation is straightforward and can be directly linked with the VRP.

00:29:05.520 After conducting some tests, I found that this approach considerably outperformed regular RPC calls, especially when it comes to responses over the network.

00:29:32.220 To conclude, while we’ve moved low-level tasks from Ruby to Go and created an efficient communication bridge using gRPC, the integration of Ruby logic with M Ruby allows us to optimize performance.

00:30:00.750 This combination demonstrates that We can successfully utilize WebSockets in Ruby, but our ambition is to achieve high performance through integrating additional languages.

00:30:36.040 Changes within Ruby, like the new Rack API, will enhance WebSocket performance in upcoming years. While memories issues remain a lasting challenge, we are hopeful that the future will yield better solutions.

00:30:56.010 I’ve already published my slides with code examples and details for further reference. I also have stickers for you!

00:31:14.950 Thank you very much for your time. We have a few minutes left for questions, so please feel free to find the microphone, turn it on, and ask your question.