00:00:16.529
Welcome to the last talk for you, or this will become, unfortunately, I hope you enjoy the conference so far.
00:00:24.449
Did you raise your hands? More fun? I know, I know you're a bit tired and the talk is going to be a little bit technical.
00:00:31.470
So, it's the last one for today, and I hope we can get through it successfully. The title of this talk is 'High-Speed Cables for Ruby.'
00:00:43.260
Probably not a very clear title; maybe we should call it 'Cables at Tiffany's,' which actually explains what we're doing right now.
00:00:53.789
I literally had some cables. Probably right now, as I understand, we do not have a live recording of this session.
00:01:00.570
Well, they're going to be kind of combined later from the PDF, okay.
00:01:08.400
I'd like to start by explaining what I have come to talk about. I'll make sure that we are on the same page.
00:01:14.900
First of all, by 'cable,' I mean a tool for building real-time applications.
00:01:22.830
Now let's transition to what a real-time application is—what is the real-time web?
00:01:30.000
We could check out Wikipedia, but that's not a good explanation. I think I can explain it a bit differently.
00:01:41.400
Real-time communication is a different type of communication from what we have in almost every web application.
00:01:47.220
Traditionally, we use request/response synchronous communication.
00:01:54.299
Request-response communication implies that in order to get some data from the server, clients have to send requests.
00:02:01.310
There's no way for the server to send data to the client without a client request.
00:02:07.799
By the way, did anyone play 'bing-bong' at the party after the first day? That was pretty fun.
00:02:13.500
That's when I actually came up with the idea for this slide: it's really a request-response model.
00:02:18.660
The server is not feeling too good in a real-time setting.
00:02:23.790
This is not real-time; it's a typical request-response cycle.
00:02:29.730
Real-time is different, and it typically differs in two main ways.
00:02:35.370
First, the communication is bi-directional, meaning messages pass both ways independently.
00:02:41.100
Second, it usually deals with persistent connections.
00:02:48.120
This could be real persistence in terms of sockets or some other form of persistence modeled by different transports.
00:02:53.460
Another aspect we should care about as backend developers—and most Ruby developers I think—is the difference in how we handle requests on the server side.
00:03:08.480
Request-response typically deals with a requests queue that follows a first-come, first-serve approach.
00:03:14.070
We have a limited number of simultaneous requests on our server.
00:03:20.760
Usually, this is a fixed amount with a queue.
00:03:27.090
In a real-time situation, we deal with a large number of concurrent requests that we have to serve.
00:03:32.239
Not all those requests are handled at once, as we could imagine in a scenario where, a century ago, there was a dinner somewhere.
00:03:38.820
There were a limited number of waiters, yet a lot of people to serve.
00:03:46.530
We have limited servers, but many people to serve, and they are served concurrently.
00:03:51.720
That's what we deal with in real-time applications.
00:03:57.890
There are many examples of real-time needs in applications, and nowadays, I don't know, raise your hand if your application uses real-time features.
00:04:04.620
About half of the audience.
00:04:09.840
This trend is increasing every day. It's a modern technology that enhances user experience.
00:04:16.950
When we talk about real-time, we usually talk about WebSockets.
00:04:23.370
We won't discuss older techniques like long polling, comments, or Flash sockets.
00:04:29.270
When we talk about WebSockets, we're also discussing concurrency.
00:04:34.670
The problem is that concurrency and Ruby do not play well together.
00:04:46.870
In the Ruby community, there's a common belief that writing real-time applications is nearly impossible due to performance concerns.
00:05:01.810
My goal for today's keynote is to help you unlearn this belief and convince you that writing concurrent applications in Ruby can be tricky, but it is possible.
00:05:14.750
You can choose a Ruby stack instead of switching to a mainstream language.
00:05:20.900
Let's quickly cover a bit about myself.
00:05:28.400
My name is Vladimir, and it's not always easy to pronounce here in the United States, so feel free to call me Vlad.
00:05:36.410
Many people know me as Belkin, which is my GitHub or Twitter handle. I'm involved in many open-source projects.
00:05:47.690
Currently, I contribute to Rails and work on improving Action Cable testing within Rails, which will be included in Rails 6.
00:05:55.540
I work as a team leader at a company called 'Evil Martians,' where we focus on product development and consultancy.
00:06:01.820
Through our commercial work, we also aim to give back to the community by extracting usable open-source tools.
00:06:09.590
Right now, I live in Brooklyn, and we have multiple bases globally, including in the United States and Russia.
00:06:15.470
If you're around and want to discuss Ruby, cables, or anything else, just reach out to me.
00:06:20.750
Finally, we've reached the point where I can start the actual discussion after our introduction.
00:06:26.690
Today, we’re going to discuss Action Cable and everything related to Real-time applications in Ruby.
00:06:32.089
So, to start, who here is using Action Cable?
00:06:38.690
That's quite a few people! Every year, more and more people are using Action Cable.
00:06:45.320
The first time I spoke about cables was almost two years ago, and there were very few people familiar with it.
00:06:52.520
I'm glad to see the growing interest in Action Cable; it's a great framework.
00:06:58.670
So, let's briefly cover what Action Cable consists of.
00:07:03.890
Action Cable provides servers that are responsible for handling WebSockets, broadcasting messages to clients, and it implements a publish/subscribe pattern.
00:07:09.170
We do not send messages to particular clients directly; rather, we use named channels called streams.
00:07:19.459
To manage everything, we have the channels framework, which acts as an abstraction layer for WebSocket connections.
00:07:27.290
It allows access to business logic for your clients.
00:07:34.370
A channel essentially functions like a controller for requests, but for WebSockets.
00:07:40.060
What's good about Action Cable is that it's really easy to build a working application in five minutes.
00:07:47.990
Yet, back five or ten years ago, people thought WebSockets were difficult; it's never been easier!
00:07:53.770
But how does this application scale? Last year's RailsConf focused on Rails scaling—Action Cable is seen as scalable by default.
00:08:01.820
We'll cover this topic because scaling in Action Cable is quite challenging.
00:08:10.000
To start with, let's look at benchmarks related to cables.
00:08:21.170
When benchmarking cables, we want to measure two things.
00:08:29.530
First: real-time characteristics, specifically how fast a cable can broadcast data to clients.
00:08:35.440
The second part is resource usage, which is critical if we're under load.
00:08:44.320
For the first benchmark, I used one conducted by a company called Hashrocket, named 'WebSocket Shootout'.
00:08:51.120
Let me explain this benchmark: Suppose we have a live comments feature on a conflicts site for live streaming.
00:08:58.480
There are thousands of people watching the keynote; of course, they want to publish comments and share their thoughts.
00:09:06.400
As one person publishes a comment, the server has to relay this comment to all other connected clients.
00:09:11.560
The time it takes for the server to do this is the latency we want to measure.
00:09:15.740
If it takes ten seconds to send out a message, it's not real-time; it’s irrelevant to the ongoing event, for example.
00:09:23.180
When we compare Action Cable with other implementations like Go or Erlang, it becomes interesting.
00:09:30.100
Action Cable doesn't perform well as the number of connected clients increases.
00:09:37.300
The latency tends to rise linearly with the increase in clients connected to the stream.
00:09:44.979
For instance, with 10,000 clients, the latency can reach as high as ten seconds.
00:09:52.160
That’s not real-time and that's not good.
00:09:59.779
This doesn't mean we should refrain from using Action Cable; it depends on your use case.
00:10:06.290
This leads me to what I call the 'Cable Theorem.'
00:10:12.510
With Action Cable, you either use uncrowded channels, meaning fewer subscribers.
00:10:18.060
If the number of subscribers exceeds a thousand, you will most likely experience high latency.
00:10:23.460
If latency isn’t a concern, then you can use Action Cable without issues.
00:10:30.010
On the downside, regardless of how you use it, resource usage will be high, particularly regarding CPU.
00:10:37.660
This is what server monitoring looks like when you’re running Action Cable under pressure.
00:10:44.720
You can see that for handling the same number of effective clients, Action Cable requires significantly more memory compared to other languages.
00:10:51.550
That covers benchmarks.
00:10:59.100
Benchmarks aren’t the best way to evaluate technology; we need some real-world stories.
00:11:06.360
Let me share a real-life story that’s much scarier than benchmarks.
00:11:13.920
There’s a project called EquipComm, which provides live statistics and translations for equestrian shows.
00:11:22.740
Every weekend, about 10,000 people tune in to this platform to watch these events.
00:11:30.320
To handle such a big audience, they initially used Heroku, which required 20 gigabytes of memory just for 10,000 clients.
00:11:37.299
Even then, they were still encountering performance problems.
00:11:45.050
What’s wrong? Why is real-world memory usage often worse than benchmarks?
00:11:52.420
I looked into this issue and made some suggestions.
00:11:58.780
So, let's discuss how we implement WebSockets with Rack-based applications.
00:12:02.260
At the core, we have this thing called Rack Hijack.
00:12:08.200
Rack is a common interface used by web applications that bridges web servers and your application.
00:12:15.690
Rack was designed for request-response communication, which is synchronous.
00:12:21.530
Around four or five years ago, a hack called Rack Hijack was introduced, allowing us to hijack the underlying socket.
00:12:30.510
This enables us to manage WebSocket communication without needing separate processes.
00:12:38.000
However, managing sockets directly affects performance and memory consumption.
00:12:44.480
Let’s take a moment to recognize two promising tools: Iodine and Lazily.
00:12:50.800
I've found there’s a limited awareness of these tools, which could significantly improve Ruby’s web server capabilities.
00:12:58.009
Iodine is a high-performance web server with unique real-time features and a custom protocol.
00:13:04.000
Lazily serves as a web framework that sits on top of Iodine, handling connections elegantly.
00:13:10.190
After benchmarking the performance, I found Iodine's capabilities are quite similar to those obtained with Go.
00:13:16.580
Both showcase impressive memory efficiency.
00:13:22.740
The way Iodine handles WebSockets could serve as a standard in future Rack implementations.
00:13:28.960
There is currently an ongoing effort to implement a pull request that offers this kind of functionality, which is quite exciting.
00:13:35.240
In summary, we need to ensure that the direction of Ruby development includes better WebSocket support.
00:13:42.720
One hypothesized issue concerns long-lived objects when dealing with Action cable connections.
00:13:49.920
I've found that many objects are retained per connection, which increases memory consumption.
00:13:56.040
My hypothesis ties back to how Ruby handles objects, which can lead to memory fragmentation.
00:14:03.800
We waste approximately 60% of allocated memory, which is concerning.
00:14:11.600
As we discuss potential improvements in Ruby, one topic that came up was Ruby 3.
00:14:17.520
There's widespread belief that generational garbage collection could assist in reducing long-lived object memory.”
00:14:24.790
This could allow us to reclaim memory more efficiently.
00:14:30.820
Compacting operations could also drastically improve performance for real-time applications.
00:14:37.920
We discussed how Threadlets could help replace processes and improve memory consumption.
00:14:45.310
Threadlets could function similarly to fibers, providing lightweight abstractions for concurrency.
00:14:52.900
Together with compression and efficient handling of long-lived objects, we could witness significant improvements.
00:14:59.780
It's all about rethinking how we approach concurrency in Ruby; we can look towards incorporating other languages.
00:15:09.280
The final part of the discussion centers on a project called AnyCable.
00:15:19.000
AnyCable represents a means to combine the strengths of languages alongside Ruby.
00:15:26.790
The core idea behind AnyCable is to allow the utilization of different languages to handle specific tasks.
00:15:33.400
You can keep most of the interesting code in Ruby while delegating intensive tasks to more performant languages.
00:15:40.490
AnyCable acts as an extension of Action Cable, providing developers flexibility.
00:15:46.900
By not tying the entire application to Action Cable, you can achieve a more scalable architecture.
00:15:54.200
In terms of setup, you run a dedicated WebSocket server alongside your Rails application.
00:16:02.700
Mapping out this architecture may feel complex initially.
00:16:08.600
However, it opens numerous opportunities for optimizing performance and resources.
00:16:14.800
AnyCable comes with the ability to track connections, allowing for optimized management.
00:16:19.790
Throughout this approach, we see a reduction in memory issues, as demonstrated by the project's success.
00:16:27.700
Also, there's support for zero-disconnect deployments, which are highly beneficial.
00:16:34.570
You can redeploy without affecting existing connections.
00:16:41.200
AnyCable effectively circumvents the connection avalanche which can occur during redeploys.
00:16:48.300
The result is a space where Ruby code continues executing amidst these changes.
00:16:54.480
We also have comprehensive analytic capabilities built into AnyCable.
00:17:01.110
You can track active connections, messages, and other pertinent metrics.
00:17:08.920
This is a game-changer for managing real-time applications more efficiently.
00:17:16.540
While we still have much work to do, the future for real-time Ruby applications looks promising.
00:17:23.480
We're increasingly seeing frameworks evolve, and the next evolution may include broader support for non-Ruby languages for specific tasks.
00:17:35.480
I encourage everyone to engage with these developments actively.
00:17:40.190
Ruby is not exclusive, and we should leverage the best tools available, regardless of the language.
00:17:48.150
The aim should be to write performant applications while enjoying the elegance that Ruby provides.
00:17:56.920
Thank you for your attention. I look forward to discussing these ideas further with you.
00:18:11.130
If you have any questions or wish to engage in further discussions, just let me know!
