Talks
Bending the Curve: How Rust Helped Us Write Better Ruby
Bending the Curve: How Rust Helped Us Write Better Ruby
by Yehuda Katz and Tom DaleThe video "Bending the Curve: How Rust Helped Us Write Better Ruby" features Yehuda Katz and Tom Dale, who explore their experience integrating Rust into their Skylight application, an analytics tool designed for Ruby applications. They address the challenges faced by small development teams competing against larger companies and highlight the need for more efficient tools to improve application performance metrics.
Key points discussed in the video include:
- Introduction to Skylight: Katz and Dale share their background and the inception of Skylight, created in response to misleading average performance metrics provided by existing tools. They recognized the need for a clearer understanding of performance through histograms rather than just averages.
- Performance Challenges with Ruby: They discuss performance limitations in Ruby that prompted the need for additional data processing capabilities.
- Introduction to Rust: After evaluating their options, Rust emerged as a strong candidate due to its combination of low-level control similar to C/C++ and high-level safety guarantees, which alleviated concerns regarding memory leaks and segmentation faults.
- Implementation of Rust: The decision to rewrite significant components of Skylight using Rust led to substantial performance improvements and allowed for features like real-time memory allocation tracking. They also discuss the introduction of new subscription services providing performance metric updates.
- Rust's Benefits: Katz and Dale emphasize several key features of Rust, including its ownership model, borrowing system, and deterministic memory management. These features contribute to safer and more efficient code, reducing common issues found in languages such as Ruby.
- Practical Examples and Anecdotes: They illustrate the advantages of Rust in maintaining stability and performance during the development of Skylight, using examples to showcase how Rust's ownership rules prevent mutable states and ensure consistent application behavior.
- Conclusion: The speakers conclude by encouraging developers to consider Rust for applications where performance and safety are priorities. They express optimism about Rust's evolving ecosystem and its increasing adoption across various domains.
This insightful session encourages developers to explore Rust as a tool for enhancing Ruby applications, especially in terms of performance monitoring and resource efficiency.
00:00:05.750
Thank you all for making the trip over to the sponsoring track. It's great to see so many of you here.
00:00:11.590
We started our company back in 2012, and we're really pleased to be here. We've been working on Skylight and the changes surrounding that.
00:00:18.730
One of the things we've heard about before is the idea of being a small team and putting code into your backpack.
00:00:25.160
When we look at the fact that we only have about eight people, competing with much larger companies is a real challenge.
00:00:38.870
As a small team with limited resources, we realized we needed higher leverage tools to match the competition.
00:00:47.320
The primary problem we wanted to solve with Skylight was the inaccuracy of average performance metrics that were being reported.
00:01:03.890
Most of the tools we used reported averages, which were misleading. We wanted to provide a clearer picture of our application's performance.
00:01:21.310
It is easy to be misled by average numbers because they can be skewed by a small number of very fast responses.
00:01:34.030
For instance, if you have a thousand requests that return in five milliseconds but also have two hundred requests taking two seconds, the average would still appear good.
00:01:52.979
This can be more than just misleading; it can be harmful, so we needed to display true performance metrics.
00:02:00.040
We envisioned a solution that would provide histograms instead of mere averages.
00:02:11.560
Looking at histograms gives us a much clearer view of performance distribution, including the 95th percentile, which helps us better understand what customers are experiencing.
00:02:27.040
By utilizing these high-leverage open-source tools, we're able to create a significantly better product.
00:02:35.290
Our back-end is built on top of a powerful open-source data processing framework called Apache Storm.
00:02:42.730
Last year, we talked about our architecture and how Skylight operates, which we hope you found enjoyable.
00:03:01.570
This year, we want to focus on a new topic: Rust.
00:03:16.900
Our goal with Skylight is to provide users with actionable insights that require minimal input from them.
00:03:31.680
To achieve this, we realized we needed to collect significantly more data than what Ruby could handle.
00:03:43.110
We encountered performance issues in Ruby that made us reconsider our approach.
00:03:50.620
We were looking for a programming language that offered low-level control, similar to C or C++, but with better safety guarantees.
00:04:02.870
We had our doubts, given that we’re not the greatest programmers, and the thought of using something like native code made us apprehensive.
00:04:16.520
But Rust provided us with the high-level safety we wanted while allowing the low-level control we needed.
00:04:28.160
We ultimately decided to rewrite major components of our Skylight agent from Ruby to Rust.
00:04:40.260
This was one of the best decisions we've made—although it was nerve-racking to place our faith in a relatively new programming language.
00:04:52.900
The result was that Skylight could now collect far more information than our competitors, all while consuming fewer resources.
00:05:06.900
This not only improved performance but also allowed us to develop features that would be impossible without that level of control.
00:05:22.490
For instance, we launched a new feature that clearly shows memory allocations in your Rails production environment.
00:05:35.580
This is crucial for tracking down performance bottlenecks within your application.
00:05:47.630
We're also announcing a new subscription feature that provides you with weekly updates, showing you key performance metrics over time.
00:06:09.060
If you're drawn to a language that offers low-level control, high-level safety, and extensive expressiveness, Rust is definitely worth considering.
00:06:22.080
Now, let's dive deeper into how Rust fits into our process.
00:06:30.680
We know that many of you have questions about the Rust programming model.
00:06:38.400
I'll start by explaining some of the benefits and ideas behind Rust.
00:06:47.870
So, let’s get started with some of the critical features that we find advantageous.
00:06:58.900
For many people, programming languages are often judged on expressiveness and speed.
00:07:10.580
I believe this can be a somewhat misguided way to evaluate languages.
00:07:26.740
For example, when I first started with JavaScript, it was often critiqued for its lack of expressiveness and speed.
00:07:43.090
However, the fact that JavaScript was ubiquitous—as the language of the browser—had a profound impact on how quickly it evolved.
00:08:01.920
The same can be said for Rust; despite it not being as widely adopted initially, its potential is significant.
00:08:16.570
The language aims to strike a balance between low-level control and high-level safety—the ideal scenario for many applications.
00:08:27.940
In a fast-moving environment, reducing the risks associated with memory management while retaining performance is crucial.
00:08:46.270
As a small, agile team, we had to navigate the complexities of developing applications while ensuring performance and safety.
00:09:05.600
That’s why Rust entered our conversation; it provided us with the necessary tools to build applications that are safe yet performant.
00:09:25.480
So, let's talk about systems programming and what it means within the context of Rust.
00:09:38.560
When I refer to systems programming, I mean the ability to write to the lowest levels of hardware while having the confidence that memory safety is guaranteed.
00:09:54.160
We see a significant shift in how Rust manages ownership and lifecycle of variables, a critical component of systems programming that hugely benefits developers.
00:10:12.840
For one, it allows you to write code that can allocate objects in a deterministic manner, without the unpredictability often present in languages like Ruby.
00:10:36.950
When you start thinking about memory allocation, particularly in performance-driven applications, it's essential to know exactly when memory is freed.
00:10:54.880
This is vital, especially for applications needing high throughput where garbage collection pauses would be detrimental.
00:11:10.800
Understanding how Rust approaches this problem can provide insight into writing better code that minimizes unexpected pauses.
00:11:24.250
Let me take a moment to explain how Rust manages memory allocation and ownership.
00:11:42.050
In Rust, every allocated object has a clear owner, meaning the scope of the variable is intuitive and well-defined.
00:12:04.870
If I transfer ownership of that object to another entity, I know that I will no longer have access to that object, which greatly helps manage program complexity.
00:12:22.780
This kind of ownership ensures that variables cannot exist beyond their scope, and reduces the possibility of dangling pointers or memory leaks.
00:12:42.170
As I transfer ownership, no one else can access that object until ownership is explicitly returned to the original owner, ensuring that violations of safety are caught at compile time.
00:13:01.670
This is a fundamental difference from how languages like Ruby manage memory, and it can significantly reduce runtime errors connected to memory management.
00:13:23.460
Moreover, Rust's borrowing system enhances safety even further, as it allows multiple parts of your program to access data simultaneously while enforcing that no modifications can occur unless explicitly allowed.
00:13:39.120
Each part of your program can ask for access, and Rust ensures that these requests don't lead to inconsistent states.
00:13:57.620
To put it simply, by using Rust, we reduce risk while still maintaining a fast and responsive application.
00:14:15.270
As we began using Rust, we saw immediate benefits in performance, especially when dealing with numerous objects and concurrent executions.
00:14:32.530
This allows us to think differently about structuring our code and creating scalability in our applications.
00:14:42.740
I hope this gives you a glimpse of why we moved toward Rust and how it could potentially benefit your projects.
00:15:00.220
Now let's dive deeper into what we have learned from Rust and how you can apply it in your own work.
00:15:12.500
Rust provides flexibility in terms of ownership and borrowing that can streamline complex changes in large codebases.
00:15:21.600
Next, I want to walk you through some specific examples of how Rust handles challenges that may arise during development.
00:15:32.700
In our case, there were examples that demonstrated the effectiveness of Rust code in various application scenarios.
00:15:47.530
For instance, let’s discuss the utility of handling memory in a Rust-based web application and how this differs from more traditional languages.
00:16:10.960
One of the common pitfalls in many languages is allowing for mutable state across different parts of the codebase.
00:16:28.390
Rust’s system prevents this by enforcing strict ownership rules that dictate how data can be accessed.
00:16:42.800
This guarantees that multiple parts of your application do not interfere with each other, which can lead to unpredictable results.
00:17:01.850
In our own implementation, we saw improvements in both stability and performance due to Rust's strict ownership model.
00:17:15.840
As we built out our Skylight features, the reliability of Rust allowed us to focus more on the overall architecture, rather than debugging memory-related issues.
00:17:29.160
Additionally, Rust’s community-driven ecosystem continues to grow, offering a variety of libraries to leverage—all of which come with built-in safety guarantees.
00:17:45.200
If you haven’t looked into Rust yet, I encourage you to explore the possibilities it offers for your projects.
00:18:00.470
It’s increasingly being adopted in places where performance and safety are paramount.
00:18:14.650
Before I conclude, I want to touch on some exciting developments happening in the Rust landscape.
00:18:27.890
The rise of success stories surrounding Rust is indicative of its robust features and adaptability to various challenges.
00:18:42.220
Much like any other programming language, it continues to mature as its community grows and contributes.
00:18:55.820
With that said, I hope today's discussion inspired you to investigate how Rust can enhance your development processes.
00:19:09.350
If you’re already using it or considering it, the possibilities are exciting and worthwhile.
00:19:25.560
Thank you for your attention, and we hope to see you exploring Rust further.
