Talks
High-Speed Cables for Ruby
Summarized using AI
High-Speed Cables for Ruby
by Vladimir DementyevIn his talk 'High-Speed Cables for Ruby' at RubyConf 2018, Vladimir Dementyev addresses the challenges and possibilities of building real-time applications using Ruby, which traditionally struggles with concurrency. He emphasizes the growing demand for real-time features in modern web applications and argues against abandoning Ruby in favor of other languages.
Key points discussed include:
- Definition of Real-Time Applications: Unlike traditional request/response communications, real-time applications utilize bi-directional communication and persistent connections, essential for efficient data exchange.
- Concurrency Issues: Ruby's limitations with concurrency have led to a belief that it's nearly impossible to develop real-time applications. Dementyev aims to counter this perception.
- Action Cable: A significant part of the discussion focuses on Action Cable, Ruby on Rails' framework for WebSockets, which allows developers to implement real-time features relatively simply. However, it faces scalability challenges.
- Performance Benchmarking: Dementyev shares benchmarks indicating that Action Cable incurs significant latency with an increasing number of connected clients, demonstrating that it may not be suitable for high-load scenarios.
- Memory Management: He highlights the memory consumption issues in Action Cable, which can lead to inefficient resource use under load, especially for long-lived connections.
- Alternative Tools: The talk introduces Iodine and Lazily as alternatives that provide better memory efficiency and concurrent handling, performing similarly to more mainstream languages like Go.
- AnyCable: Dementyev presents AnyCable as a solution to combine Ruby with more performant languages, allowing developers to delegate intensive tasks, thereby enhancing scalability and performance without discarding Ruby’s elegance.
In conclusion, Dementyev encourages Ruby developers to embrace new tools and languages to optimize their real-time applications while maintaining the elegance that Ruby offers. As the framework and support for real-time Ruby applications evolve, developers should stay engaged with these advancements for better performance. The talk underscores the importance of innovation within the Ruby ecosystem in the context of concurrent application development.
00:00:16.529
Welcome to the last talk for you, or this will become, unfortunately, I hope you enjoy the conference so far.
00:00:24.449
Did you raise your hands? More fun? I know, I know you're a bit tired and the talk is going to be a little bit technical.
00:00:31.470
So, it's the last one for today, and I hope we can get through it successfully. The title of this talk is 'High-Speed Cables for Ruby.'
00:00:43.260
Probably not a very clear title; maybe we should call it 'Cables at Tiffany's,' which actually explains what we're doing right now.
00:00:53.789
I literally had some cables. Probably right now, as I understand, we do not have a live recording of this session.
00:01:00.570
Well, they're going to be kind of combined later from the PDF, okay.
00:01:08.400
I'd like to start by explaining what I have come to talk about. I'll make sure that we are on the same page.
00:01:14.900
First of all, by 'cable,' I mean a tool for building real-time applications.
00:01:22.830
Now let's transition to what a real-time application is—what is the real-time web?
00:01:30.000
We could check out Wikipedia, but that's not a good explanation. I think I can explain it a bit differently.
00:01:41.400
Real-time communication is a different type of communication from what we have in almost every web application.
00:01:47.220
Traditionally, we use request/response synchronous communication.
00:01:54.299
Request-response communication implies that in order to get some data from the server, clients have to send requests.
00:02:01.310
There's no way for the server to send data to the client without a client request.
00:02:07.799
By the way, did anyone play 'bing-bong' at the party after the first day? That was pretty fun.
00:02:13.500
That's when I actually came up with the idea for this slide: it's really a request-response model.
00:02:18.660
The server is not feeling too good in a real-time setting.
00:02:23.790
This is not real-time; it's a typical request-response cycle.
00:02:29.730
Real-time is different, and it typically differs in two main ways.
00:02:35.370
First, the communication is bi-directional, meaning messages pass both ways independently.
00:02:41.100
Second, it usually deals with persistent connections.
00:02:48.120
This could be real persistence in terms of sockets or some other form of persistence modeled by different transports.
00:02:53.460
Another aspect we should care about as backend developers—and most Ruby developers I think—is the difference in how we handle requests on the server side.
00:03:08.480
Request-response typically deals with a requests queue that follows a first-come, first-serve approach.
00:03:14.070
We have a limited number of simultaneous requests on our server.
00:03:20.760
Usually, this is a fixed amount with a queue.
00:03:27.090
In a real-time situation, we deal with a large number of concurrent requests that we have to serve.
00:03:32.239
Not all those requests are handled at once, as we could imagine in a scenario where, a century ago, there was a dinner somewhere.
00:03:38.820
There were a limited number of waiters, yet a lot of people to serve.
00:03:46.530
We have limited servers, but many people to serve, and they are served concurrently.
00:03:51.720
That's what we deal with in real-time applications.
00:03:57.890
There are many examples of real-time needs in applications, and nowadays, I don't know, raise your hand if your application uses real-time features.
00:04:04.620
About half of the audience.
00:04:09.840
This trend is increasing every day. It's a modern technology that enhances user experience.
00:04:16.950
When we talk about real-time, we usually talk about WebSockets.
00:04:23.370
We won't discuss older techniques like long polling, comments, or Flash sockets.
00:04:29.270
When we talk about WebSockets, we're also discussing concurrency.
00:04:34.670
The problem is that concurrency and Ruby do not play well together.
00:04:46.870
In the Ruby community, there's a common belief that writing real-time applications is nearly impossible due to performance concerns.
00:05:01.810
My goal for today's keynote is to help you unlearn this belief and convince you that writing concurrent applications in Ruby can be tricky, but it is possible.
00:05:14.750
You can choose a Ruby stack instead of switching to a mainstream language.
00:05:20.900
Let's quickly cover a bit about myself.
00:05:28.400
My name is Vladimir, and it's not always easy to pronounce here in the United States, so feel free to call me Vlad.
00:05:36.410
Many people know me as Belkin, which is my GitHub or Twitter handle. I'm involved in many open-source projects.
00:05:47.690
Currently, I contribute to Rails and work on improving Action Cable testing within Rails, which will be included in Rails 6.
00:05:55.540
I work as a team leader at a company called 'Evil Martians,' where we focus on product development and consultancy.
00:06:01.820
Through our commercial work, we also aim to give back to the community by extracting usable open-source tools.
00:06:09.590
Right now, I live in Brooklyn, and we have multiple bases globally, including in the United States and Russia.
00:06:15.470
If you're around and want to discuss Ruby, cables, or anything else, just reach out to me.
00:06:20.750
Finally, we've reached the point where I can start the actual discussion after our introduction.
00:06:26.690
Today, we’re going to discuss Action Cable and everything related to Real-time applications in Ruby.
00:06:32.089
So, to start, who here is using Action Cable?
00:06:38.690
That's quite a few people! Every year, more and more people are using Action Cable.
00:06:45.320
The first time I spoke about cables was almost two years ago, and there were very few people familiar with it.
00:06:52.520
I'm glad to see the growing interest in Action Cable; it's a great framework.
00:06:58.670
So, let's briefly cover what Action Cable consists of.
00:07:03.890
Action Cable provides servers that are responsible for handling WebSockets, broadcasting messages to clients, and it implements a publish/subscribe pattern.
00:07:09.170
We do not send messages to particular clients directly; rather, we use named channels called streams.
00:07:19.459
To manage everything, we have the channels framework, which acts as an abstraction layer for WebSocket connections.
00:07:27.290
It allows access to business logic for your clients.
00:07:34.370
A channel essentially functions like a controller for requests, but for WebSockets.
00:07:40.060
What's good about Action Cable is that it's really easy to build a working application in five minutes.
00:07:47.990
Yet, back five or ten years ago, people thought WebSockets were difficult; it's never been easier!
00:07:53.770
But how does this application scale? Last year's RailsConf focused on Rails scaling—Action Cable is seen as scalable by default.
00:08:01.820
We'll cover this topic because scaling in Action Cable is quite challenging.
00:08:10.000
To start with, let's look at benchmarks related to cables.
00:08:21.170
When benchmarking cables, we want to measure two things.
00:08:29.530
First: real-time characteristics, specifically how fast a cable can broadcast data to clients.
00:08:35.440
The second part is resource usage, which is critical if we're under load.
00:08:44.320
For the first benchmark, I used one conducted by a company called Hashrocket, named 'WebSocket Shootout'.
00:08:51.120
Let me explain this benchmark: Suppose we have a live comments feature on a conflicts site for live streaming.
00:08:58.480
There are thousands of people watching the keynote; of course, they want to publish comments and share their thoughts.
00:09:06.400
As one person publishes a comment, the server has to relay this comment to all other connected clients.
00:09:11.560
The time it takes for the server to do this is the latency we want to measure.
00:09:15.740
If it takes ten seconds to send out a message, it's not real-time; it’s irrelevant to the ongoing event, for example.
00:09:23.180
When we compare Action Cable with other implementations like Go or Erlang, it becomes interesting.
00:09:30.100
Action Cable doesn't perform well as the number of connected clients increases.
00:09:37.300
The latency tends to rise linearly with the increase in clients connected to the stream.
00:09:44.979
For instance, with 10,000 clients, the latency can reach as high as ten seconds.
00:09:52.160
That’s not real-time and that's not good.
00:09:59.779
This doesn't mean we should refrain from using Action Cable; it depends on your use case.
00:10:06.290
This leads me to what I call the 'Cable Theorem.'
00:10:12.510
With Action Cable, you either use uncrowded channels, meaning fewer subscribers.
00:10:18.060
If the number of subscribers exceeds a thousand, you will most likely experience high latency.
00:10:23.460
If latency isn’t a concern, then you can use Action Cable without issues.
00:10:30.010
On the downside, regardless of how you use it, resource usage will be high, particularly regarding CPU.
00:10:37.660
This is what server monitoring looks like when you’re running Action Cable under pressure.
00:10:44.720
You can see that for handling the same number of effective clients, Action Cable requires significantly more memory compared to other languages.
00:10:51.550
That covers benchmarks.
00:10:59.100
Benchmarks aren’t the best way to evaluate technology; we need some real-world stories.
00:11:06.360
Let me share a real-life story that’s much scarier than benchmarks.
00:11:13.920
There’s a project called EquipComm, which provides live statistics and translations for equestrian shows.
00:11:22.740
Every weekend, about 10,000 people tune in to this platform to watch these events.
00:11:30.320
To handle such a big audience, they initially used Heroku, which required 20 gigabytes of memory just for 10,000 clients.
00:11:37.299
Even then, they were still encountering performance problems.
00:11:45.050
What’s wrong? Why is real-world memory usage often worse than benchmarks?
00:11:52.420
I looked into this issue and made some suggestions.
00:11:58.780
So, let's discuss how we implement WebSockets with Rack-based applications.
00:12:02.260
At the core, we have this thing called Rack Hijack.
00:12:08.200
Rack is a common interface used by web applications that bridges web servers and your application.
00:12:15.690
Rack was designed for request-response communication, which is synchronous.
00:12:21.530
Around four or five years ago, a hack called Rack Hijack was introduced, allowing us to hijack the underlying socket.
00:12:30.510
This enables us to manage WebSocket communication without needing separate processes.
00:12:38.000
However, managing sockets directly affects performance and memory consumption.
00:12:44.480
Let’s take a moment to recognize two promising tools: Iodine and Lazily.
00:12:50.800
I've found there’s a limited awareness of these tools, which could significantly improve Ruby’s web server capabilities.
00:12:58.009
Iodine is a high-performance web server with unique real-time features and a custom protocol.
00:13:04.000
Lazily serves as a web framework that sits on top of Iodine, handling connections elegantly.
00:13:10.190
After benchmarking the performance, I found Iodine's capabilities are quite similar to those obtained with Go.
00:13:16.580
Both showcase impressive memory efficiency.
00:13:22.740
The way Iodine handles WebSockets could serve as a standard in future Rack implementations.
00:13:28.960
There is currently an ongoing effort to implement a pull request that offers this kind of functionality, which is quite exciting.
00:13:35.240
In summary, we need to ensure that the direction of Ruby development includes better WebSocket support.
00:13:42.720
One hypothesized issue concerns long-lived objects when dealing with Action cable connections.
00:13:49.920
I've found that many objects are retained per connection, which increases memory consumption.
00:13:56.040
My hypothesis ties back to how Ruby handles objects, which can lead to memory fragmentation.
00:14:03.800
We waste approximately 60% of allocated memory, which is concerning.
00:14:11.600
As we discuss potential improvements in Ruby, one topic that came up was Ruby 3.
00:14:17.520
There's widespread belief that generational garbage collection could assist in reducing long-lived object memory.”
00:14:24.790
This could allow us to reclaim memory more efficiently.
00:14:30.820
Compacting operations could also drastically improve performance for real-time applications.
00:14:37.920
We discussed how Threadlets could help replace processes and improve memory consumption.
00:14:45.310
Threadlets could function similarly to fibers, providing lightweight abstractions for concurrency.
00:14:52.900
Together with compression and efficient handling of long-lived objects, we could witness significant improvements.
00:14:59.780
It's all about rethinking how we approach concurrency in Ruby; we can look towards incorporating other languages.
00:15:09.280
The final part of the discussion centers on a project called AnyCable.
00:15:19.000
AnyCable represents a means to combine the strengths of languages alongside Ruby.
00:15:26.790
The core idea behind AnyCable is to allow the utilization of different languages to handle specific tasks.
00:15:33.400
You can keep most of the interesting code in Ruby while delegating intensive tasks to more performant languages.
00:15:40.490
AnyCable acts as an extension of Action Cable, providing developers flexibility.
00:15:46.900
By not tying the entire application to Action Cable, you can achieve a more scalable architecture.
00:15:54.200
In terms of setup, you run a dedicated WebSocket server alongside your Rails application.
00:16:02.700
Mapping out this architecture may feel complex initially.
00:16:08.600
However, it opens numerous opportunities for optimizing performance and resources.
00:16:14.800
AnyCable comes with the ability to track connections, allowing for optimized management.
00:16:19.790
Throughout this approach, we see a reduction in memory issues, as demonstrated by the project's success.
00:16:27.700
Also, there's support for zero-disconnect deployments, which are highly beneficial.
00:16:34.570
You can redeploy without affecting existing connections.
00:16:41.200
AnyCable effectively circumvents the connection avalanche which can occur during redeploys.
00:16:48.300
The result is a space where Ruby code continues executing amidst these changes.
00:16:54.480
We also have comprehensive analytic capabilities built into AnyCable.
00:17:01.110
You can track active connections, messages, and other pertinent metrics.
00:17:08.920
This is a game-changer for managing real-time applications more efficiently.
00:17:16.540
While we still have much work to do, the future for real-time Ruby applications looks promising.
00:17:23.480
We're increasingly seeing frameworks evolve, and the next evolution may include broader support for non-Ruby languages for specific tasks.
00:17:35.480
I encourage everyone to engage with these developments actively.
00:17:40.190
Ruby is not exclusive, and we should leverage the best tools available, regardless of the language.
00:17:48.150
The aim should be to write performant applications while enjoying the elegance that Ruby provides.
00:17:56.920
Thank you for your attention. I look forward to discussing these ideas further with you.
00:18:11.130
If you have any questions or wish to engage in further discussions, just let me know!
Explore all talks recorded at RubyConf 2018
+82