Guild Prototype

by Koichi Sasada

The video titled "Guild Prototype" is presented by Koichi Sasada at RubyKaigi 2018. In this talk, Sasada discusses progress on a concurrency abstraction named Guild for Ruby 3, originally proposed two years prior. The Guild aims to provide a safer and more efficient model for concurrent programming by minimizing issues associated with traditional thread-based paradigms. The video highlights key points about Guild and its prototype, including:

Background of Guild: The Guild concept is a response to the difficulties Ruby programmers face with thread-based programming, particularly related to synchronization and race conditions.
Productivity Focus: The aim is to enhance programmer productivity by simplifying the complexity of multi-threading, as modern hardware increasingly relies on multi-core processing.
Key Features of Guild: Each Guild operates without shared mutable state, thus avoiding many bugs associated with traditional shared-object programming. This promotes independent operation and reduces the risk of race conditions.
Demonstration of Guild: Sasada showcases a benchmark comparing traditional single-threaded Fibonacci calculations with a Guild-based implementation. The use of multiple Guilds dramatically speeds up the computation, with Guilds allowing better CPU resource utilization. For instance, a process that takes 30 minutes with 40 Guilds might take over three hours on a single-threaded approach.
Scalability Concerns: While the Guild model offers efficiency improvements with scalability, the overhead of managing small tasks can sometimes offset performance gains due to the complexity introduced by managing multiple Guilds.
Other Applications: Sasada also tests Guild with tasks such as counting unique elements in a file, displaying its versatility in handling various computational tasks effectively.
Future Improvements: He expresses a desire to further optimize Guild for better performance and stability, as well as to simplify the synchronization mechanisms involved.
Naming Discussion: Finally, the presentation touches on the significance of naming the abstraction, discussing potential rebranding to reflect its objectives better.

In conclusion, Sasada emphasizes the potential of Guild to revolutionize concurrency in Ruby, making it more accessible to developers. He invites participants to engage further during the conference and explore the Guild prototype available at the Cookpad booth for hands-on experience.

00:00:00.060 Hello everyone, my name is Koichi Sasada. Thank you for attending this presentation.

00:00:06.680 Today's presentation is titled 'Guild Prototype'. Two years ago, I spoke about a new concurrency abstraction model for Ruby 3.

00:00:16.049 At that time, I proposed an idea named 'Guild'. Today, I will show you the progress of this project.

00:00:30.810 The summary of this presentation is that we can run programs with Guild, not just on my laptop or my environment. However, it is still very buggy because it is a thread-based programming model.

00:00:47.520 Thread programming can be quite challenging, especially for me, and perhaps for you as well. The key phrase here is that programming with threads is difficult, and today, I will cover these topics.

00:01:07.409 I want to introduce the background of Guild, demonstrate this orientation, and provide a little of my background in this project. I work with the Ruby interpreter project and continue to develop the MRI implementation.

00:01:37.409 Recently, with the release of Ruby 2.6, significant improvements were made, making it 1.4 times faster without breaking existing features.

00:02:02.329 Additionally, I'm proud to be an employee of Cookpad. I encourage you to visit our booth if you're interested in trying it out. We also have global office hours today and a Q&A session tomorrow.

00:02:41.909 Before we dive deeper, I want to briefly mention that we are hosting a party tonight. If you'd like to join, please visit Cookpad's booth at 6:40 PM.

00:03:16.870 Now, let’s discuss the background of Guild. The most important aspect for Ruby programmers is productivity. We have many tasks to complete, and often we are quite busy.

00:03:39.720 Our goal is to make programming easier, especially for concurrency programming, which is notoriously difficult. While Ruby makes it easy to create threads, it is challenging to write safe concurrent programs using thread primitives because we must manage synchronization.

00:04:22.599 As we continue to utilize more CPUs, such as those available in modern laptops and servers, it becomes increasingly important to simplify the complexity of multi-threaded programming.

00:05:13.950 This is where the concept of Guild comes in. The key idea is to avoid sharing mutable state between Guilds. In traditional thread-based programming, sharing objects is easy, but doing so introduces numerous race conditions and bugs.

00:06:10.021 Guild is designed to address these issues and allow for effective concurrent programming by ensuring that each Guild operates independently without shared state.

00:06:54.669 Therefore, it can facilitate performance improvements. Instead of sharing objects, which leads to complications, each Guild would have its objects, and interaction between them would be managed through well-defined interfaces.

00:07:55.000 I would like to demonstrate the current implementation of Guild using this machine. This machine has 2 CPUs, each with 10 cores, totaling 40 logical CPUs.

00:08:42.000 Before I proceed, let me show you the simple program that calculates Fibonacci numbers. A single-threaded implementation without Guild is straightforward.

00:09:06.350 With the Guild-based version, I created a Manager Guild that manages several worker Guilds, distributing work among them. This allows us to better utilize the available processing power and significantly speed up execution.

00:10:22.659 As we increase the number of Guilds working, we can see a drastic improvement in speed. For example, the task that takes only 30 minutes with 40 Guilds would take over three hours with just one conventional Ruby script.

00:10:47.400 We can conclude that when scaling workloads with Guilds, the efficiency improves drastically as more Guilds are introduced to share the workload.

00:11:06.600 However, we see diminishing returns for very small tasks, as the overhead for managing Guilds can exceed the performance gain from parallelization.

00:12:12.630 I also explored other tasks, such as counting unique elements in a file with Guild, demonstrating that the application is versatile and can handle various computational challenges.

00:12:55.150 We see that while the result is straightforward, its efficiency shows favorable comparisons to traditional implementations, showcasing Guild's potential.

00:13:38.150 Moving forward, I aim to refine Guild's architecture further. My goal is to enhance performance and stability while minimizing overhead and complexity introduced by the Guild model.

00:14:23.310 To this end, I am working on optimizing the system to handle complex operations safely and efficiently, delving into synchronization strategies that can benefit effective resource management.

00:14:59.460 Finally, I would like to touch upon the naming of this project. The term 'Guild' reflects the group-like structure this model promotes, but I recognize that names are important.

00:15:53.770 There has been discussion on whether to rename this abstraction to better convey its intention and application in the language’s evolution.

00:16:40.670 Thank you all for your attention. I am excited about the future of Guild, and I welcome your questions during this conference.

00:17:04.730 Feel free to reach out to me personally or visit our booth for more information. Thank you!