Talks
Letting Concurrency Help You Today
Summarized using AI
Letting Concurrency Help You Today
by Richard BishopThe video "Letting Concurrency Help You Today" by Richard Bishop, presented at RubyConf 2014, delves into the often misunderstood topic of concurrency in programming, particularly focusing on its significance for Ruby developers. Bishop begins by defining concurrency and distinguishing it from parallelism, highlighting that concurrency is about managing multiple tasks concurrently, while parallelism involves executing multiple tasks simultaneously.
Key Points Discussed:
- Importance of Concurrency: Bishop emphasizes the need for concurrency in a multi-core world, arguing that maximizing hardware resources can improve performance and user experience. He notes that better designs often stem from considering concurrency in application development.
- Basic Concepts: The talk covers foundational concepts of concurrency, including processes, threads, and events, explaining how they relate to communication and parallelism in programming.
- Ruby's Concurrency Challenges: Bishop addresses the limitations placed by Ruby’s Global Interpreter Lock (GIL) and the cost of forking processes, which complicates effective concurrency utilization.
- Practical Applications: He introduces several libraries and tools to enhance concurrency in Ruby applications:
- Celluloid: A library that implements the actor model for concurrency, allowing for communication through messages between lightweight, concurrent objects.
- Sidekiq: A background processing tool that uses Celluloid to efficiently manage jobs in a multi-threaded environment.
- Sucker Punch: An in-memory job processing tool suitable for prototyping.
- Puma: A multi-threaded web server that integrates concurrency in its architecture.
- Design Benefits: The video discusses how concurrent programming leads to better software designs by promoting modularity and flexibility. Bishop compares Ruby's approach to concurrency with Elixir, a language designed with concurrency as a core principle, showcasing the advantages of concurrency-oriented design.
- Future Directions: Bishop mentions ongoing efforts in the Ruby community to enhance concurrency through better abstractions and improved garbage collection, signaling potential growth in Ruby’s concurrency capabilities.
Conclusions and Takeaways:
- Concurrency is fundamentally about coordination and communication in programming, which can lead to more efficient and scalable applications.
- Understanding and utilizing concurrency can vastly improve software design and performance in Ruby applications.
- The Ruby community is evolving towards better abstraction and support for concurrency, promising a brighter future for Ruby developers looking to harness these capabilities.
00:00:18.600
Hello, I'm Richard Bishop, and I'm here to talk to you about concurrency. Specifically, I'll discuss what concurrency is, what it isn't, and hopefully, provide you with some actionable insights for your applications today.
00:00:25.400
A little bit more about me: I’m a self-proclaimed polyglot with a problem. I love programming languages.
00:00:31.640
I don't know if it's appropriate to say this at RubyConf, but if you're not exploring another language to stretch your thinking in different ways, you really should. There's a lot of interesting stuff happening right now, particularly in how different languages tackle concurrency.
00:00:36.840
These experiences have significantly impacted me as a Rubyist. Away from the keyboard, I’m also a big coffee and beer enthusiast, and it's great that this event is in San Diego because I used to live here. In my opinion, we also have the best beer in the world, so if you come across some local brews, definitely give them a try. Just make sure you don't indulge too much, as you are at a conference, and you don't want to make a fool of yourself.
00:00:45.360
Since this is a conference, I should give a shout-out to my employer: I'm a freelance programmer. We are not hiring.
00:00:50.840
For me, learning about concurrency was somewhat accidental. I like to refer to it as 'the other side of Rack.' Most of our time is spent on the application call side, while on the completely opposite side, we navigate through web servers and system calls.
00:00:58.719
One night, I encountered an issue with bind2, and at the time, I was wondering, 'What's bind2?' It looked like just another method call, and I wasn’t even fully aware of what a system call was. This curiosity led me deeper into understanding the patterns that web servers like Unicorn implement.
00:01:05.880
It turned out to be a lot of fun, and Ruby is actually a great language to explore concurrency because it provides all the necessary primitives. That said, it does leave you wishing for more, which is part of the story of Ruby and concurrency.
00:01:12.240
So, let’s dive into what I'm covering today. We will start with some basic concepts of concurrency. Some of you might already be familiar with a lot of this, while for others, it might be brand new.
00:01:19.360
We’ll also look at practical uses of concurrency in Ruby, focusing on abstractions because that's really what concurrency is about.
00:01:24.640
While this is a Ruby conference, about 80% of this talk will be dedicated to general concurrency, and the remaining 20% will be how it relates specifically to Ruby.
00:01:32.320
Concurrency is a vast and complex topic. To kick things off, I'll start with the basics.
00:01:39.560
There’s a common exercise in concurrency talks that involves spawning threads and measuring how quickly you can populate an array with a million integers. But honestly, that never captured my interest in concurrency. It’s important to differentiate between concurrency and parallelism, and we’ll talk about those differences later. I’m not going to critique the Global Interpreter Lock (GIL) because that’s a common topic and it's here, so we have to deal with it.
00:01:50.760
Most of our work is I/O bound anyway, and I'm not going to delve into the other runtimes because we, as Rubyists, are pretty spoiled with our ability to run our code on several runtimes.
00:01:56.000
However, each of these runtimes could each represent a talk of its own.
00:02:06.759
So, why should we care about concurrency? We’ve all heard the adage that 'the free lunch is over,' and we now live in a multi-core world. We need to make better use of our hardware.
00:02:12.959
As Rubyists, we know that forking is quite expensive. If you don’t receive the server bill every month, you might not care, but I believe we should be maximizing the potential of our hardware.
00:02:18.800
Another reason is better user experience. If you examine highly concurrent, parallel runtimes, they consistently deliver stable response times, avoiding those extreme 99th percentiles and long-tail latencies.
00:02:24.879
You can’t just consider requests per second or average response times; it’s essential to look at the entire picture.
00:02:31.599
One major surprise for me while learning about concurrency was noticing that better designs arise from designing programs concurrently. Now let’s delve into what concurrency actually is.
00:02:37.879
First, it’s important to clarify this point. Concurrency often gets confused with parallelism, but these concepts are actually quite different.
00:02:42.920
Thankfully, in recent years, a clear definition has emerged, courtesy of Rob Pike, one of the creators of Go. He outlined that concurrency is about dealing with many things at once, whereas parallelism is about doing many things at once.
00:02:48.319
That's a solid definition, but if you’re like me, upon first encountering it, you might understand what concurrency isn’t, but you still won’t fully grasp what it is.
00:02:54.640
To make this more concrete, let’s consider application development. If you're working on a multicore computer using threads or a multitude of processes, those can indeed be CPU-bound up to the number of cores you have running at the same time.
00:03:01.599
However, the important thing is that those are merely points of execution; there’s no communication occurring among them.
00:03:06.720
In the context of a web server, a request acts as a boundary for concurrency. A prime example would be batch processing: even on something like MRI, you could fire up 30 Sidekiq processors. They may not all consume CPU time simultaneously, but they are indeed running concurrently.
00:03:12.360
When you group tasks together, there can be communication present. For instance, if you have a parent job, it can update its status to indicate success or failure or trigger additional actions once all tasks are completed.
00:03:19.480
So, the reason there’s so much focus on concurrency today is that we desire the benefits of parallelism, but we want a clear and concise approach to achieve it.
00:03:24.879
Ultimately, concurrency is about composition and communication; it sits at the nexus of performance and clarity. That’s why there are so many various models to approach it.
00:03:30.439
Before diving deeper, I came across an interesting tweet while preparing this presentation that I think is worth sharing. It stated that programmers love two things: making generalizations and dismissing other people's generalizations since they are often outliers.
00:03:37.760
I’m mentioning this because concurrency is an expansive topic; within 45 minutes, I can't possibly cover everything there is to know. You may have had experiences that don’t align with what I'm sharing today.
00:03:44.319
In covering the basics of concurrency, there are three essential concepts provided by operating systems and hardware developers: processes, threads, and events.
00:03:50.280
These are the foundational elements that you’ll encounter. If we examine these in the context of parallelism, on a multicore machine, processes allow for both CPU and I/O parallelism.
00:03:55.360
Threads offer the same benefits, but events, which are kernel events, are solely designated for I/O; there’s no CPU concurrency or parallelism associated with events.
00:04:01.200
The last column in this context pertains to communication, which, again, is what concurrency is fundamentally about. When numerous processes are operating, it’s crucial to establish a means of communication among them.
00:04:06.920
With processes, communication is handled via inter-process communication on the same machine, which can involve pipes, sockets, or message queues. That’s the sole way for them to relay information.
00:04:12.880
In contrast, with threads, we share memory to facilitate communication, which introduces its own set of challenges. Unfortunately, events by themselves do not offer a communication path since callbacks are simply executed as they complete.
00:04:18.840
Hence, the need for substantial abstraction when dealing with events; hence the jab at Node.js. Alone, events can become quite complex.
00:04:24.280
In other languages like Clojure and Erlang, the handling of events is usually concealed by the runtime, which simplifies interactions considerably.
00:04:30.560
In summary, concurrency revolves around composition and communication. It’s how your programs are structured versus how they are executed simultaneously.
00:04:36.240
In this talk, we will focus mainly on threads within Ruby, which have been designated as the fundamental unit of concurrency.
00:04:42.720
We have not fully embraced this concept yet, thus lacking great support for threading in Ruby. However, if we engage with threads correctly—meaning not using them directly, or not being aware we’re utilizing them—the experience becomes significantly better.
00:04:49.760
When I consider threading and concurrency, it resembles several other complex challenges we’ve faced as developers. The essence is that something should be difficult to accomplish, yet important enough to pursue, and if that’s the case, we should tackle it head-on.
00:04:55.400
Let’s take an example: memory management. We are all familiar—especially this year—with issues surrounding manual memory management, like the Heartbleed vulnerability.
00:05:01.700
However, this process was abstracted long ago; Lisp was known as the first garbage-collected language, and Ruby, of course, utilizes garbage collection.
00:05:06.960
Most of our work occurs in garbage-collected environments, and there are languages like Rust that have an innovative ownership model addressing memory management.
00:05:13.280
The key takeaway is that we must manage memory effectively, and while it is not necessary to manually handle it, understanding how it works is important.
00:05:19.440
Of course, we encounter SQL as well—who doesn’t love the occasional SQL injection?
00:05:25.920
In Ruby, we have query generators and ORM (like Active Record) to help mitigate these issues for us.
00:05:32.560
And when it comes to threading, some languages offer excellent abstractions, but Ruby unfortunately does not.
00:05:39.920
When comparing memory management to threading, I think of threads and mutexes as akin to malloc and free. We should be aware that they are in use somewhere, but ideally not have to deal with them daily.
00:05:47.040
In Ruby, we certainly love processes. We take joy in forking and utilizing all that RAM, but this comes with high costs.
00:05:54.280
Communicating effectively between several processes can be challenging. I refer to this form of communication as passive-aggressive because one process sends information to a message queue without knowing which other process will ultimately receive that message.
00:06:00.320
Without direct communication, it's difficult to construct truly robust and composable systems.
00:06:07.160
Fortunately, due to our community, there are some libraries available today that we can utilize to ease these challenges. I want to highlight a few, with hopes of motivating you to start using them.
00:06:14.440
The first library I'd like to discuss is Celluloid. Developed by Tony Arcieri, it's an excellent library for concurrency.
00:06:20.000
If you’re familiar with the actor model, as seen in languages like Erlang and Elixir, Celluloid implements this model well. It allows you to harness concurrency primitives in a familiar manner, where actors execute concurrently and communicate through messages.
00:06:26.760
Some people argue that methods are messages in a way, but I’ll show how this view can be inaccurate later on.
00:06:33.160
Celluloid integrates the actor model into Ruby objects, providing a cohesive library for concurrency.
00:06:39.200
Interestingly, this has become one of the most downloaded gems that many users may not even be aware they are using, as it’s employed by several other outstanding libraries.
00:06:45.320
The library also includes handy built-in features like pooling actor objects for work distribution and supervisors designed for fault tolerance and reliability.
00:06:51.600
To illustrate an actor in Celluloid, you can envision it having a mailbox in which messages are stored. The mailbox functions as an advanced queue that processes incoming messages in order.
00:06:58.120
When messages are retrieved from the queue, the actor processes them and proceeds with its functionality.
00:07:04.560
In Celluloid, the message contains the method to invoke and the respective arguments to provide.
00:07:10.240
Here’s a simple, albeit contrived, example of using Celluloid. When mixing it in, you can instantiate an object.
00:07:16.480
Notice that the result isn't the defined class but rather a wrapped instance inside a Celluloid proxy, through which you can communicate by sending messages asynchronously.
00:07:22.000
You utilize the async method to get back an async proxy, allowing you to call methods like greet on it.
00:07:28.080
However, it’s vital to emphasize that method calls aren't messages; this distinction is crucial.
00:07:34.440
Method calls are synchronous by nature and must wait for completion before proceeding.
00:07:40.560
One way to bypass this delay is to spawn a thread to 'fire and forget.' However, this approach is not something recommended for frequent use.
00:07:46.760
In the actor model, messages can be both asynchronous and synchronous. You can opt to require a reply after sending a message, but the initial state remains asynchronous.
00:07:53.160
Celluloid implements this functionality through metaprogramming. When invoking async along with a method, the async proxy carries out method_missing, wrapping the call and forwarding it for later processing.
00:07:59.760
The reason I wanted to highlight this library first is that the next one relies on it, and I hope everyone here is already using it: Sidekiq.
00:08:05.440
Sidekiq is a phenomenal background processing library developed by Mike Perham. In my experience, one Sidekiq worker can replace between 8 to 20 classic Resque workers.
00:08:11.480
This efficiency stems from its multi-threaded nature, offering great benefits even in MRI because of I/O parallelism.
00:08:18.360
Examining Sidekiq’s design reveals that it utilizes Celluloid behind the scenes, featuring a launcher object that initializes fetchers and managers.
00:08:24.920
The manager oversees these processors, which are responsible for instantiating and executing your jobs. This clear separation of concerns is remarkable.
00:08:31.160
All these elements operate concurrently, fostering effective communication through messages. The manager is able to send requests to the job processor and the fetcher sends tasks to the manager.
00:08:37.600
This leads me to another question: who here is currently utilizing Sidekiq? That’s a good number of you!
00:08:44.040
Ruby is popular among startups, and I understand that startups often feel incredibly busy, leaving little time to consider concurrency. This brings me to another intriguing background job library called Sucker Punch.
00:08:51.520
Developed by Brandon Hilyer, it also uses Celluloid and operates entirely in memory, sharing the same API as Sidekiq for easy upgrades.
00:08:58.320
However, Sucker Punch is not recommended for crucial tasks since it is purely in-memory; if your process crashes, the queue is lost.
00:09:05.840
Nonetheless, it’s quite effective for prototyping and similar scenarios. The last library I’d like to discuss is Puma, a multi-threaded web server.
00:09:12.240
Puma supports forking and includes a reactor, offering a comprehensive concurrency model.
00:09:18.920
Examining Puma’s design reveals a clear separation of concerns. It features a thread pool, a popular concurrency abstraction with shared resources among threads, guarded by a mutex around a request queue.
00:09:25.600
On the contrary, an acceptor thread handles TCP requests and transfers them to a request queue that gets processed by the threads.
00:09:32.720
This design fosters straightforward separation of concerns, essential for effective threading. However, potential bottlenecks exist, with one being the mutex synchronizing access to the queue.
00:09:39.520
Fortunately, this isn't a major concern because it’s often overshadowed by the real bottleneck within Ruby itself.
00:09:46.560
To recap this section, everything we've discussed revolves around abstractions. You've seen actors and thread pools, and I hope I’ve encouraged you to explore libraries that can enhance your efficiency significantly.
00:09:52.960
The next segment delves into what I've found particularly fascinating about concurrency: the design benefits that arise when you leverage concurrent programming.
00:09:59.840
In today’s landscape, there’s an enormous emphasis on software design—many discussions revolve around concepts like designing in small scopes, service-oriented architecture, service objects, or microservices.
00:10:06.720
Essentially, what design boils down to is the pain experienced when working on a single-threaded synchronous application with poor structure; the resulting unexpected bugs and challenges in making changes lead to slower iterations.
00:10:14.840
Conversely, when dealing with a concurrent application, flawed design introduces not just the same issues, but also deadlocks, race conditions, and additional complications.
00:10:21.600
Recognizing this enables you to adopt a more merciless approach to your design process.
00:10:26.960
I want to briefly introduce you to another language called Elixir. While I can't thoroughly cover its intricacies at this moment, I’d like to provide some high-level insights.
00:10:34.720
Elixir, created by José Valim, runs on the Erlang virtual machine. If you've met an Erlang programmer, they've likely extolled the numerous advantages of the Erlang VM within moments of conversation.
00:10:40.880
The striking aspect of both Elixir and Erlang is their paradigmatic approach to concurrency. They treat concurrency as a standalone paradigm, much like object-oriented or functional programming.
00:10:48.080
Within these languages, you'll find structured applications defined by supervision trees, where individual components run concurrently and communicate through message passing.
00:10:54.000
What’s notable is that the red circles, known as supervisors, manage these actors. These structural distinctions enhance flexibility and adaptability in your designs.
00:11:00.160
When crafting Elixir applications, you begin by categorizing your library code—your models, parsing logic—and progressively implementing the concurrently running actors.
00:11:06.960
This method increases scalability rates tenfold. In a typical four or eight-core machine, one can run 300,000 to 1,000,000 actors concurrently.
00:11:13.200
In the Erlang world, while some naming conventions may be obscure, this structure greatly enhances modularity. Each actor, or process as they refer to them, can be independently deployed.
00:11:19.440
In contrast to the Rails world, which often revolves around monolithic deployments, this modular design allows for granular scalability.
00:11:25.920
With a traditional Ruby application, a large component could hinder deployments as scaling typically involves deploying the entire code base. However, in the Erlang context, individual components like caches can be upgraded effortlessly.
00:11:32.080
A common challenge in Ruby applications is balancing simplicity in code structure against the complexities arising from deployment strategies.
00:11:39.680
We typically favor simplicity during design, but this often leads to convoluted deployment structures, with multiple processes necessitating varying commands.
00:11:46.280
Elixir’s architecture contrasts with Ruby's historical approach. The simplistic view of single-threaded code conceals a complex deployment reality that involves multiple processes.
00:11:53.520
Thus, while single-threaded synchronous programs might seem easier to manage, they introduce complexities that have a cascading effect on deployment configurations.
00:11:59.840
To reiterate the elegance of Elixir's concurrent design, all components are executed concurrently within a single code base, enabling individual scaling.
00:12:06.640
This modular approach is desirable in any programming scenario, especially when seeking to leverage the advantages of concurrent design.
00:12:13.720
In Ruby, there’s often a tendency to simulate that kind of scalable design, often resulting in various services that complicate our operational landscape.
00:12:20.360
When deploying services, several teams often create redundant systems that might only slightly partition a monolithic codebase.
00:12:27.600
A prevalent notion is that you need services to handle individual aspects because your Ruby application is deployed too frequently.
00:12:34.000
However, what ends up happening is that you cobble together message queues and provide HTTP communication between services that were meant to be decoupled.
00:12:40.640
In practice, this often leads to what I would describe as repository-oriented architecture, with a haphazard web of interdependencies.
00:12:47.920
While SOA is often justified by the promise of clean separations and individual scaling, the process of using message queues often complicates these aspirations.
00:12:54.720
With messages crossing various components through HTTP, you can find yourself coupling services unnecessarily over the network.
00:13:01.920
To further recapitulate, concurrent design mandates that you start with modular architecture; it’s a fundamental requirement in languages like Elixir and Erlang.
00:13:08.240
With adequate scaling, independent deployment becomes possible for each actor or process, allowing teams to adjust units without compromising a central application.
00:13:15.200
In contrast, designing for concurrency in Ruby often leads to intricate service-oriented architectures where pulling one thread can unravel the entire structure.
00:13:22.240
The final aspect I want to cover relates to discussing the Ruby concurrency model, particularly as it pertains to the GIL.
00:13:28.960
While I haven't criticized the GIL yet, I do want to highlight that JRuby is an excellent option if GIL is a significant issue for you.
00:13:35.680
To comprehend Ruby's structure, we should acknowledge its design history dating back to the '90s. I wasn’t programming back then, but looking back, one can see the patterns.
00:13:42.360
Many languages from that time share common traits. A programming language often starts off by selecting a paradigm, like object-oriented programming, of which Matt chose a beneficial version.
00:13:49.480
Towards the end of that design phase, someone slaps on some basic threads and considers that the concurrency model. Ruby had green threads from the outset, albeit not really built for concurrency.
00:13:56.640
In that period, concurrency didn’t even register on most coding radars, hence the GIL's existence.
00:14:04.880
Flash-forward to the 2000s and beyond, you notice a shift. Languages like Go actively prioritize concurrency right from their design.
00:14:11.520
The design is such that steps toward concurrency are inherently integrated, involving aspects like immutability that frame the core model.
00:14:18.520
In Ruby, the perception is that concurrency and parallelism should mesh seamlessly, as if Matt were to simply flip a switch.
00:14:25.760
However, the reality is that other elements require addressing first; primarily, a competent garbage collector.
00:14:32.680
If concurrency and parallelism are introduced without an efficient garbage collector, you risk stalling due to world stops and facing significant performance issues.
00:14:39.040
Currently, considerable efforts are underway to improve Ruby's garbage collector, as seen with versions 2.1 and 2.2.
00:14:46.520
Secondly, we need to develop abstractions beyond the current, limited threading and mutexes we have in Ruby today.
00:14:54.400
Currently, the core provides basic tools like threads, mutexes, and condition variables, falling short compared to many other languages.
00:15:02.800
Rubyists often find themselves envious when comparing their threading structures against those in languages like Java.
00:15:10.520
One ultimate factor influencing Ruby's concurrency struggles is its culture of simplicity. We leverage Ruby because it eases our programming tasks and allows for enjoyable coding experiences.
00:15:17.640
However, much of the current concurrency discussion is rooted in cutting-edge theories, often dating back to the 70s and 80s, which frequently lack clear implementation guidance.
00:15:25.560
Languages like Go have teams with extensive Unix programming experience, leading to a sound implementation process with multiple iterations based on learned experiences.
00:15:32.680
As a result, the journey toward implementing actionable concurrency models is intricate and demands time for exploration.
00:15:39.520
Matt recently expressed a commitment to introducing abstract concurrency units such as actors, signaling that a change may be on the horizon.
00:15:47.160
The community can anticipate the arrival of improved abstractions soon—when they emerge, they will provide the needed simplicity alongside advanced capabilities.
00:15:54.680
That concludes my talk. Thank you for your attention!
Explore all talks recorded at RubyConf 2014
+73