High-Speed Cables for Ruby

by Vladimir Dementyev

In his talk 'High-Speed Cables for Ruby' at RubyConf 2018, Vladimir Dementyev addresses the challenges and possibilities of building real-time applications using Ruby, which traditionally struggles with concurrency. He emphasizes the growing demand for real-time features in modern web applications and argues against abandoning Ruby in favor of other languages.

Key points discussed include:
- Definition of Real-Time Applications: Unlike traditional request/response communications, real-time applications utilize bi-directional communication and persistent connections, essential for efficient data exchange.
- Concurrency Issues: Ruby's limitations with concurrency have led to a belief that it's nearly impossible to develop real-time applications. Dementyev aims to counter this perception.
- Action Cable: A significant part of the discussion focuses on Action Cable, Ruby on Rails' framework for WebSockets, which allows developers to implement real-time features relatively simply. However, it faces scalability challenges.
- Performance Benchmarking: Dementyev shares benchmarks indicating that Action Cable incurs significant latency with an increasing number of connected clients, demonstrating that it may not be suitable for high-load scenarios.
- Memory Management: He highlights the memory consumption issues in Action Cable, which can lead to inefficient resource use under load, especially for long-lived connections.
- Alternative Tools: The talk introduces Iodine and Lazily as alternatives that provide better memory efficiency and concurrent handling, performing similarly to more mainstream languages like Go.
- AnyCable: Dementyev presents AnyCable as a solution to combine Ruby with more performant languages, allowing developers to delegate intensive tasks, thereby enhancing scalability and performance without discarding Ruby’s elegance.

In conclusion, Dementyev encourages Ruby developers to embrace new tools and languages to optimize their real-time applications while maintaining the elegance that Ruby offers. As the framework and support for real-time Ruby applications evolve, developers should stay engaged with these advancements for better performance. The talk underscores the importance of innovation within the Ruby ecosystem in the context of concurrent application development.

00:00:16.529 Welcome to the last talk for you, or this will become, unfortunately, I hope you enjoy the conference so far.

00:00:24.449 Did you raise your hands? More fun? I know, I know you're a bit tired and the talk is going to be a little bit technical.

00:00:31.470 So, it's the last one for today, and I hope we can get through it successfully. The title of this talk is 'High-Speed Cables for Ruby.'

00:00:43.260 Probably not a very clear title; maybe we should call it 'Cables at Tiffany's,' which actually explains what we're doing right now.

00:00:53.789 I literally had some cables. Probably right now, as I understand, we do not have a live recording of this session.

00:01:00.570 Well, they're going to be kind of combined later from the PDF, okay.

00:01:08.400 I'd like to start by explaining what I have come to talk about. I'll make sure that we are on the same page.

00:01:14.900 First of all, by 'cable,' I mean a tool for building real-time applications.

00:01:22.830 Now let's transition to what a real-time application is—what is the real-time web?

00:01:30.000 We could check out Wikipedia, but that's not a good explanation. I think I can explain it a bit differently.

00:01:41.400 Real-time communication is a different type of communication from what we have in almost every web application.

00:01:47.220 Traditionally, we use request/response synchronous communication.

00:01:54.299 Request-response communication implies that in order to get some data from the server, clients have to send requests.

00:02:01.310 There's no way for the server to send data to the client without a client request.

00:02:07.799 By the way, did anyone play 'bing-bong' at the party after the first day? That was pretty fun.

00:02:13.500 That's when I actually came up with the idea for this slide: it's really a request-response model.

00:02:18.660 The server is not feeling too good in a real-time setting.

00:02:23.790 This is not real-time; it's a typical request-response cycle.

00:02:29.730 Real-time is different, and it typically differs in two main ways.

00:02:35.370 First, the communication is bi-directional, meaning messages pass both ways independently.

00:02:41.100 Second, it usually deals with persistent connections.

00:02:48.120 This could be real persistence in terms of sockets or some other form of persistence modeled by different transports.

00:02:53.460 Another aspect we should care about as backend developers—and most Ruby developers I think—is the difference in how we handle requests on the server side.

00:03:08.480 Request-response typically deals with a requests queue that follows a first-come, first-serve approach.

00:03:14.070 We have a limited number of simultaneous requests on our server.

00:03:20.760 Usually, this is a fixed amount with a queue.

00:03:27.090 In a real-time situation, we deal with a large number of concurrent requests that we have to serve.

00:03:32.239 Not all those requests are handled at once, as we could imagine in a scenario where, a century ago, there was a dinner somewhere.

00:03:38.820 There were a limited number of waiters, yet a lot of people to serve.

00:03:46.530 We have limited servers, but many people to serve, and they are served concurrently.

00:03:51.720 That's what we deal with in real-time applications.

00:03:57.890 There are many examples of real-time needs in applications, and nowadays, I don't know, raise your hand if your application uses real-time features.

00:04:04.620 About half of the audience.

00:04:09.840 This trend is increasing every day. It's a modern technology that enhances user experience.

00:04:16.950 When we talk about real-time, we usually talk about WebSockets.

00:04:23.370 We won't discuss older techniques like long polling, comments, or Flash sockets.

00:04:29.270 When we talk about WebSockets, we're also discussing concurrency.

00:04:34.670 The problem is that concurrency and Ruby do not play well together.

00:04:46.870 In the Ruby community, there's a common belief that writing real-time applications is nearly impossible due to performance concerns.

00:05:01.810 My goal for today's keynote is to help you unlearn this belief and convince you that writing concurrent applications in Ruby can be tricky, but it is possible.

00:05:14.750 You can choose a Ruby stack instead of switching to a mainstream language.

00:05:20.900 Let's quickly cover a bit about myself.

00:05:28.400 My name is Vladimir, and it's not always easy to pronounce here in the United States, so feel free to call me Vlad.

00:05:36.410 Many people know me as Belkin, which is my GitHub or Twitter handle. I'm involved in many open-source projects.

00:05:47.690 Currently, I contribute to Rails and work on improving Action Cable testing within Rails, which will be included in Rails 6.

00:05:55.540 I work as a team leader at a company called 'Evil Martians,' where we focus on product development and consultancy.

00:06:01.820 Through our commercial work, we also aim to give back to the community by extracting usable open-source tools.

00:06:09.590 Right now, I live in Brooklyn, and we have multiple bases globally, including in the United States and Russia.

00:06:15.470 If you're around and want to discuss Ruby, cables, or anything else, just reach out to me.

00:06:20.750 Finally, we've reached the point where I can start the actual discussion after our introduction.

00:06:26.690 Today, we’re going to discuss Action Cable and everything related to Real-time applications in Ruby.

00:06:32.089 So, to start, who here is using Action Cable?

00:06:38.690 That's quite a few people! Every year, more and more people are using Action Cable.

00:06:45.320 The first time I spoke about cables was almost two years ago, and there were very few people familiar with it.

00:06:52.520 I'm glad to see the growing interest in Action Cable; it's a great framework.

00:06:58.670 So, let's briefly cover what Action Cable consists of.

00:07:03.890 Action Cable provides servers that are responsible for handling WebSockets, broadcasting messages to clients, and it implements a publish/subscribe pattern.

00:07:09.170 We do not send messages to particular clients directly; rather, we use named channels called streams.

00:07:19.459 To manage everything, we have the channels framework, which acts as an abstraction layer for WebSocket connections.

00:07:27.290 It allows access to business logic for your clients.

00:07:34.370 A channel essentially functions like a controller for requests, but for WebSockets.

00:07:40.060 What's good about Action Cable is that it's really easy to build a working application in five minutes.

00:07:47.990 Yet, back five or ten years ago, people thought WebSockets were difficult; it's never been easier!

00:07:53.770 But how does this application scale? Last year's RailsConf focused on Rails scaling—Action Cable is seen as scalable by default.

00:08:01.820 We'll cover this topic because scaling in Action Cable is quite challenging.

00:08:10.000 To start with, let's look at benchmarks related to cables.

00:08:21.170 When benchmarking cables, we want to measure two things.

00:08:29.530 First: real-time characteristics, specifically how fast a cable can broadcast data to clients.

00:08:35.440 The second part is resource usage, which is critical if we're under load.

00:08:44.320 For the first benchmark, I used one conducted by a company called Hashrocket, named 'WebSocket Shootout'.

00:08:51.120 Let me explain this benchmark: Suppose we have a live comments feature on a conflicts site for live streaming.

00:08:58.480 There are thousands of people watching the keynote; of course, they want to publish comments and share their thoughts.

00:09:06.400 As one person publishes a comment, the server has to relay this comment to all other connected clients.

00:09:11.560 The time it takes for the server to do this is the latency we want to measure.

00:09:15.740 If it takes ten seconds to send out a message, it's not real-time; it’s irrelevant to the ongoing event, for example.

00:09:23.180 When we compare Action Cable with other implementations like Go or Erlang, it becomes interesting.

00:09:30.100 Action Cable doesn't perform well as the number of connected clients increases.

00:09:37.300 The latency tends to rise linearly with the increase in clients connected to the stream.

00:09:44.979 For instance, with 10,000 clients, the latency can reach as high as ten seconds.

00:09:52.160 That’s not real-time and that's not good.

00:09:59.779 This doesn't mean we should refrain from using Action Cable; it depends on your use case.

00:10:06.290 This leads me to what I call the 'Cable Theorem.'

00:10:12.510 With Action Cable, you either use uncrowded channels, meaning fewer subscribers.

00:10:18.060 If the number of subscribers exceeds a thousand, you will most likely experience high latency.

00:10:23.460 If latency isn’t a concern, then you can use Action Cable without issues.

00:10:30.010 On the downside, regardless of how you use it, resource usage will be high, particularly regarding CPU.

00:10:37.660 This is what server monitoring looks like when you’re running Action Cable under pressure.

00:10:44.720 You can see that for handling the same number of effective clients, Action Cable requires significantly more memory compared to other languages.

00:10:51.550 That covers benchmarks.

00:10:59.100 Benchmarks aren’t the best way to evaluate technology; we need some real-world stories.

00:11:06.360 Let me share a real-life story that’s much scarier than benchmarks.

00:11:13.920 There’s a project called EquipComm, which provides live statistics and translations for equestrian shows.

00:11:22.740 Every weekend, about 10,000 people tune in to this platform to watch these events.

00:11:30.320 To handle such a big audience, they initially used Heroku, which required 20 gigabytes of memory just for 10,000 clients.

00:11:37.299 Even then, they were still encountering performance problems.

00:11:45.050 What’s wrong? Why is real-world memory usage often worse than benchmarks?

00:11:52.420 I looked into this issue and made some suggestions.

00:11:58.780 So, let's discuss how we implement WebSockets with Rack-based applications.

00:12:02.260 At the core, we have this thing called Rack Hijack.

00:12:08.200 Rack is a common interface used by web applications that bridges web servers and your application.

00:12:15.690 Rack was designed for request-response communication, which is synchronous.

00:12:21.530 Around four or five years ago, a hack called Rack Hijack was introduced, allowing us to hijack the underlying socket.

00:12:30.510 This enables us to manage WebSocket communication without needing separate processes.

00:12:38.000 However, managing sockets directly affects performance and memory consumption.

00:12:44.480 Let’s take a moment to recognize two promising tools: Iodine and Lazily.

00:12:50.800 I've found there’s a limited awareness of these tools, which could significantly improve Ruby’s web server capabilities.

00:12:58.009 Iodine is a high-performance web server with unique real-time features and a custom protocol.

00:13:04.000 Lazily serves as a web framework that sits on top of Iodine, handling connections elegantly.

00:13:10.190 After benchmarking the performance, I found Iodine's capabilities are quite similar to those obtained with Go.

00:13:16.580 Both showcase impressive memory efficiency.

00:13:22.740 The way Iodine handles WebSockets could serve as a standard in future Rack implementations.

00:13:28.960 There is currently an ongoing effort to implement a pull request that offers this kind of functionality, which is quite exciting.

00:13:35.240 In summary, we need to ensure that the direction of Ruby development includes better WebSocket support.

00:13:42.720 One hypothesized issue concerns long-lived objects when dealing with Action cable connections.

00:13:49.920 I've found that many objects are retained per connection, which increases memory consumption.

00:13:56.040 My hypothesis ties back to how Ruby handles objects, which can lead to memory fragmentation.

00:14:03.800 We waste approximately 60% of allocated memory, which is concerning.

00:14:11.600 As we discuss potential improvements in Ruby, one topic that came up was Ruby 3.

00:14:17.520 There's widespread belief that generational garbage collection could assist in reducing long-lived object memory.”

00:14:24.790 This could allow us to reclaim memory more efficiently.

00:14:30.820 Compacting operations could also drastically improve performance for real-time applications.

00:14:37.920 We discussed how Threadlets could help replace processes and improve memory consumption.

00:14:45.310 Threadlets could function similarly to fibers, providing lightweight abstractions for concurrency.

00:14:52.900 Together with compression and efficient handling of long-lived objects, we could witness significant improvements.

00:14:59.780 It's all about rethinking how we approach concurrency in Ruby; we can look towards incorporating other languages.

00:15:09.280 The final part of the discussion centers on a project called AnyCable.

00:15:19.000 AnyCable represents a means to combine the strengths of languages alongside Ruby.

00:15:26.790 The core idea behind AnyCable is to allow the utilization of different languages to handle specific tasks.

00:15:33.400 You can keep most of the interesting code in Ruby while delegating intensive tasks to more performant languages.

00:15:40.490 AnyCable acts as an extension of Action Cable, providing developers flexibility.

00:15:46.900 By not tying the entire application to Action Cable, you can achieve a more scalable architecture.

00:15:54.200 In terms of setup, you run a dedicated WebSocket server alongside your Rails application.

00:16:02.700 Mapping out this architecture may feel complex initially.

00:16:08.600 However, it opens numerous opportunities for optimizing performance and resources.

00:16:14.800 AnyCable comes with the ability to track connections, allowing for optimized management.

00:16:19.790 Throughout this approach, we see a reduction in memory issues, as demonstrated by the project's success.

00:16:27.700 Also, there's support for zero-disconnect deployments, which are highly beneficial.

00:16:34.570 You can redeploy without affecting existing connections.

00:16:41.200 AnyCable effectively circumvents the connection avalanche which can occur during redeploys.

00:16:48.300 The result is a space where Ruby code continues executing amidst these changes.

00:16:54.480 We also have comprehensive analytic capabilities built into AnyCable.

00:17:01.110 You can track active connections, messages, and other pertinent metrics.

00:17:08.920 This is a game-changer for managing real-time applications more efficiently.

00:17:16.540 While we still have much work to do, the future for real-time Ruby applications looks promising.

00:17:23.480 We're increasingly seeing frameworks evolve, and the next evolution may include broader support for non-Ruby languages for specific tasks.

00:17:35.480 I encourage everyone to engage with these developments actively.

00:17:40.190 Ruby is not exclusive, and we should leverage the best tools available, regardless of the language.

00:17:48.150 The aim should be to write performant applications while enjoying the elegance that Ruby provides.

00:17:56.920 Thank you for your attention. I look forward to discussing these ideas further with you.

00:18:11.130 If you have any questions or wish to engage in further discussions, just let me know!