Talks
Speakers
Events
Topics
Sign in
Home
Talks
Speakers
Events
Topics
Leaderboard
Use
Analytics
Sign in
Suggest modification to this talk
Title
Description
RubyConf 2019 - Disk is fast, memory is slow. Forget all you think you know by Daniel Magliola Adding metrics to your code should effectively have no impact on performance. When we were recently tasked with doing that in multi-process Ruby servers, we ran into an interesting challenge: could we aggregate our numbers across processes without blowing our target of just one microsecond of overhead? The months of work that followed had us looking into C extensions, segfault dumps, memory maps, syscall timings, and pretty much everything we could think of to try and achieve our objective. In the process, we found some very counter-intuitive performance results that I'd like to share with you. #confreaks #rubyconf2019
Date
Summarized using AI?
If this talk's summary was generated by AI, please check this box. A "Summarized using AI" badge will be displayed in the summary tab to indicate that the summary was generated using AI.
Show "Summarized using AI" badge on summary page
Summary
Markdown supported
The video titled "Disk is fast, memory is slow. Forget all you think you know" by Daniel Magliola explores performance considerations in Ruby applications, particularly in the context of monitoring metrics using Prometheus in multi-process environments. Magliola outlines several critical challenges faced by developers when attempting to gather aggregate performance metrics across multiple processes due to Ruby's Global Interpreter Lock (GIL) and the use of prefork servers. He discusses the following key points: - **Overview of the Problem**: Magliola introduces Go-Catalyst's need for reliable metrics in their banking application where uptime is critical. He highlights the use of Prometheus for monitoring but points out the complications arising from multi-process architecture. - **Multiple Processes and Memory Sharing**: Since Ruby processes do not share memory, each maintains its own counters, complicating the metric aggregation required for Prometheus’s functionality. Magliola examines the limitations of existing Ruby libraries in addressing this concurrency issue. - **Performance Targets**: The need to maintain performance metrics with minimal overhead led to the goal of incrementing counters within a microsecond. He shares insights about the goal’s challenging nature, indicating skepticism from peers about achieving it in Ruby. - **Experimentation with Solutions**: Magliola investigates multiple approaches, including the use of hashes, various storage methods, and Redis for data storage and inter-process communication. Notably, he addresses the performance constraints faced when deploying memory maps and the difficulties of using C extensions in Ruby. - **Surprising Outcomes**: Ultimately, through a series of experiments, he discovers that writing to disk offers surprising performance benefits over traditional in-memory operations or more complex systems. The use of disk was revealed to be faster than expected due to the asynchronous nature of operating systems in handling write operations. - **Takeaway Messages**: The main conclusion emphasizes the importance of questioning preconceptions about performance, advocating that experimenting with different solutions can reveal unexpected efficiencies. Magliola suggests that assumptions can lead to overlooking viable paths, and encourages viewers to measure performance before optimizing assumptions. In summary, this talk illustrates how context can redefine performance assumptions, ultimately guiding developers toward discovering unconventional yet effective solutions through empirical experimentation.
Suggest modifications
Cancel
