00:00:00.080
foreign
00:00:23.920
Jeff is one of those presenters I've seen on the conference circuit. The one advantage of doing conferences regularly is that I get to meet great speakers.
00:00:30.560
You did present in Austin, right? I've seen him present at RubyConf, and one interesting part of his early presentations was that he has been doing software development full-time for zero years.
00:00:36.320
So, zero years! Awesome! But he came to software development after being a high school teacher. He now runs a company called Jumpstart Lab.
00:00:42.399
I'm going to let him talk a little more about himself, but we're glad to have him here. He did a great job with the workshops for us over the last two days as well.
00:01:17.360
Good morning! I'll try to move quickly to get you to lunch and so forth. The presentation I have for you today is about metrics-driven Ruby on Rails performance.
00:01:23.119
It's a topic I've become increasingly interested in over the last two years. I think it's one of the pieces that we're currently missing in our community.
00:01:28.560
Ruby has been famous, or perhaps infamous, for emphasizing testing over the past several years. However, performance considerations are the next step beyond testing. Testing proves that your application works, but that's not enough—it also needs to work quickly.
00:01:35.119
We're developing a growing consciousness around this need. Let's talk about how to get started with performance metrics.
00:01:41.439
As Kobe mentioned, my name is Jeff Casimir. Jumpstart Lab is my company, and with me today is Steve, who will be speaking later.
00:01:48.159
In terms of the percentage of Jumpstart Lab employees speaking here, it's 100%. We're taking over—watch out, L.A.!
00:01:54.320
In March, I was contractually engaged with the board associated with LivingSocial, where I'll be teaching a group of 24 people who know somewhere between little and nothing about programming.
00:02:04.880
After five months, they will join the LivingSocial engineering team. It's an experiment in long-term, intense developer training.
00:02:11.280
If you’re interested in this, we're planning another session in September, and I can tell you more about it during the day.
00:02:20.080
Today, we're going to discuss runtime performance—how does your app perform in production? This can be very different than during development.
00:02:29.360
One key mistake people make when interested in performance is to create sample data in development, run their code, and everything seems fine.
00:02:36.959
However, in production, having a thousand users is often a lot slower than just five users or whatever small number you used for testing.
00:02:45.760
We rarely automate the process of creating large amounts of test data. How often in your app do you think, 'This works well enough, so let me write a rake task to generate 10,000 objects and see how it feels?'
00:02:53.440
Examining runtime performance is a fascinating issue, and what you'll often find is that your database is the problem.
00:03:06.159
Ruby community members often quote, 'Optimization is unimportant,' emphasizing that if you worry about performance, you're not using Ruby correctly.
00:03:12.959
However, the full quote warns against premature optimization, which raises the question of when optimization becomes necessary.
00:03:20.080
What point is your software considered mature enough that it must perform well? It's only when a poor user is trying to use it that you need to care about performance.
00:03:30.080
The Ruby movement started around the concept of joy. You hear speakers often discuss the joy of Ruby programming. The joy of using software is when it operates as expected and does so quickly.
00:03:43.440
If software functions as intended but is slow, that does not bring joy. Similarly, if it runs quickly but does not meet your needs, it also leads to dissatisfaction.
00:03:51.200
Mature software is only a concept when people are actively using it. Users really are the ones who care about performance, while developers often overlook this aspect. However, they should prioritize user experience.
00:04:04.360
The greater your application's performance, the happier your users will be. An interesting article from either Fred Wilson or Y Combinator highlighted how the performance of applications impacts user enjoyment very early in the interaction.
00:04:15.120
When applications respond quickly, users enjoy the experience more. The more responsive your application, the more users you will gain. More users also lead to increased revenue.
00:04:26.560
Ultimately, performance translates into user experience. If I conducted a survey asking how many of you consider yourselves experienced in user experience, I bet only about ten of you would raise your hands.
00:04:40.160
This lack of recognition is a shame. We need to take the stigma off of UX; it isn’t just about being a social media expert. UX is a critical aspect, and as developers, you are among the most important contributors to user experience.
00:04:52.640
Every line of code you write directly influences user experience. You need to perceive yourself as a user experience developer or expert, acknowledging that it's the only thing that truly matters.
00:05:06.720
If your users dislike your application, no amount of beautiful patterns or plugins, or sophisticated testing frameworks will matter. They will simply represent a waste of time.
00:05:17.920
Let's discuss goal-setting. When you work with larger companies, like Etsy, they employ impressive metrics. They can chart everything from deploys to sales increases, helping analyze the impact of changes.
00:05:29.440
Etsy's scale makes it easier for them to test these ideas. For example, with a million transactions a day, even a small change can yield significant results.
00:05:38.240
However, if you have just three transactions a day, it’s challenging to simply obtain that fourth one. Etsy has advantages in scale, allowing them to directly correlate all aspects of their work with customer results.
00:05:49.760
When it comes to online shopping, like with Etsy or eBay, if a page takes more than half a second to load, users often navigate away, forgetting about the initial tab in the process.
00:06:02.560
This situation can lead to lost sales, where simply improving performance just a little bit would have made a tangible difference.
00:06:14.640
When you begin your performance metrics, aim for a threshold of 500 milliseconds: if a server-side response takes longer than this, it is considerably slow.
00:06:21.920
You may not notice the delay immediately during development since hitting refresh appears almost instant. But in production, the delays can feel painfully significant.
00:06:30.560
Matt Ayman, who shared my room, pointed out that server responses should ideally be under 100 milliseconds. Once you transmit over 500 milliseconds in user-side response, you're likely to experience noticeable delays.
00:06:45.440
If the delay is over 100 milliseconds, users will perceive your application as sluggish. It's essential to measure and focus on these metrics right from the beginning.
00:06:52.560
There's a common misconception that performance problems can always be fixed later. While this is true to a certain extent, if you wait too long, it can become incredibly challenging.
00:07:04.360
This sparked my interest in tracking performance more seriously. One excellent resource for this is New Relic.
00:07:11.280
I didn't use New Relic for a while, but after returning to it, I found it incredibly useful. I'm not affiliated with them, but I sincerely recommend their tool.
00:07:17.920
If your application generates revenue and you're not using New Relic, you're likely missing out on a valuable performance analysis tool.
00:07:26.720
New Relic provides a transparent way to dive into your code, allowing you to see issues regarding database load directly. This visibility can lead you to identify problem areas quickly.
00:07:36.160
Another vital aspect we need to discuss is performance analysis more in-depth. When looking for performance metrics, you may take a more scientific approach.
00:07:44.960
For effective performance tracking, we need to analyze CPU, memory, and database usage.
00:07:52.000
Let’s get started with CPU analysis by using PerfTools.rb. This tool, developed by Google, serves as a CPU monitoring utility that runs alongside your executed code, gathering information over time.
00:08:01.360
If someone mentions C programming, I tend to feel a bit anxious about having to write in it, but thankfully, Aman writes the C code so we don't have to.
00:08:08.960
Aman is one of my favorite personalities in the Ruby community. If you're interested in performance, his contributions are invaluable, and you will find them throughout various tools.
00:08:16.560
He has wrapped PerfTools with Ruby, enabling us to continue working in our comfort zone while still taking advantage of performance insights.
00:08:30.240
PerfTools measures every 10 milliseconds, checking what method is being executed and retrieving its stack trace.
00:08:38.320
This tool answers two burning questions: which methods are consuming time, and it's intriguing to check who calls these methods.
00:08:46.080
While the functions you write might not be CPU-intensive, they could be calling other methods that are either very slow or repeatedly executed, impacting overall performance.
00:08:56.320
PerfTools operates as a command-line program and can also be integrated into a Rack middleware. Thus, you have a convenient option, and you don’t even need to access the terminal to use it.
00:09:06.320
To get started, you just need to install the middleware and set it up in your Bundler, configuring an initializer with whatever output format you'd prefer.
00:09:12.440
If you append 'profile=true' to your URL, you’ll see performance data instead of the page content.
00:09:18.560
This data will often reveal concerning statistics, for example, that a significant portion of execution time is consumed by the garbage collector or ActiveRecord.
00:09:25.840
To accurately interpret the output produced, it’s helpful to understand that PerfTools doesn't categorize information efficiently, often presenting raw numbers without clear label headings.
00:09:32.640
You'll see on the left the sample count, but what often matters more is the rightmost column, indicating the total time percentage attributed to a method and those it calls.
00:09:39.360
This illustrates the overall responsibility for CPU cycles taken by a given method. It shows the impact of not only its cycles but also those of methods it invokes.
00:09:46.480
If you produce a PDF output, it will richly visualize the information, displaying complex graphs that illuminate your performance metrics.
00:09:54.080
If you apply it while using Rails in Rack middleware, you'll observe countless middleware dependencies and processes being executed.
00:10:01.600
These stacks can provide insightful visualizations that help you pinpoint which methods are taking excessive time and where potential bottlenecks reside.
00:10:08.320
The text outputs can be beneficial for identifying CPU-heavy methods, while the PDF format excels at displaying call stacks in a clear manner.
00:10:14.960
Let me show you an example from my sample project. This is a simple blog, which I built during a workshop, where I established a thousand articles.
00:10:21.680
The front page features a dashboard displaying five recent articles and comments, alongside some statistics.
00:10:28.160
I intentionally wrote this application to be inefficient, so I would have relevant metrics to analyze performance.
00:10:35.280
Interestingly, I found that the methods I designed to be inefficient weren't actually as slow as I anticipated.
00:10:42.960
One method computes word count by querying all articles from the database, splitting down their words, and summing them up. I thought this would be slow, considering I had a thousand articles.
00:10:50.080
Surprisingly, it executed in just a quarter of a second; however, the dashboard still took nearly two and a half seconds, which is excessive.
00:10:56.760
When running the text output, I discovered that SQL Lite and garbage collection were the primary contributors to that delay.
00:11:03.520
This insight indicated that I was likely executing many small queries, not just a few large ones, causing significant object creation.
00:11:10.480
It’s common to find scenarios where hundreds of queries are needed, creating numerous temporary objects that quickly become garbage.
00:11:17.920
As I analyzed the data, my goal was to identify what I authored and trace back through the stack to locate potential performance issues.
00:11:25.680
Among the most active methods implemented was one intended to find the most popular articles based on comments.
00:11:32.320
Initially, I aimed for a counter cache to demonstrate its significance, but I soon discovered it consumed an astonishing 60% of the runtime.
00:11:38.880
That revelation shocked me! I had no idea it would be that significant. After identifying the issue, my first step was to avoid trying to fix it immediately.
00:11:45.920
Instead, I attempted to validate that I had correctly pinpointed the source of the issue.
00:11:52.960
To simplify, I replaced the operation designed to find the most popular articles with the straightforward command to grab just one article using 'article.first'.
00:11:59.520
As expected, the execution time dropped drastically from 2300 milliseconds to 600 milliseconds. While 600 milliseconds is still not ideal, that's a significant improvement.
00:12:05.600
This highlighted a 70% reduction in runtime. My next plan was to tackle the cause of the performance issue.
00:12:12.240
PerfTools provides a quick way to profile the CPU usage of specific methods. You can supply it with a block to focus on particular elements within your application's performance.
00:12:19.440
Running it as a standalone can streamline the process; you can apply it directly to the method, excluding all the middleware and other unnecessary executions.
00:12:25.200
Evaluating the CPU time spent revealed visibly that the bottleneck resided within the database interactions.
00:12:32.000
This finding implied I was making excessive database queries, and optimizing this interaction could lead to further performance gains.
00:12:39.200
The insights from profiling the CPU performance as well as memory and databases can guide you on fixing inefficiencies.
00:12:46.640
If you take away one concept today, let it be that PerfTools.rb is incredibly powerful and you should try it out.
00:12:54.440
Many of the performance tracking tools can feel intimidating, but PerfTools allows you to stay within Ruby while still utilizing these advanced techniques.
00:13:01.520
Next, let’s discuss memory profiling. One of the first gems that comes to mind is the library 'memprof', which tracks the use of objects.
00:13:08.560
Although it’s a fantastic tool for version 1.8.7 of Ruby, it may need to be left behind as newer versions are adopted.
00:13:15.680
The latest capabilities around profiling object allocations can still be enjoyed through PerfTools.
00:13:22.560
By appending ‘mode=objects’ in your URL, you can focus on tracking object allocation instead.
00:13:31.680
The output provided can show you which methods allocate the greatest number of objects, although analyzing this data can be complex.
00:13:39.920
Finding actionable improvements can sometimes feel more challenging compared to the clarity provided with CPU profiling.
00:13:45.440
When you encounter results of allocating an extraordinarily high number of objects, it can lead to thoughtful optimization strategies.
00:13:52.520
For example, if you notice that you're creating many hashes, each with multiple string keys, consider switching to symbols instead.
00:13:58.560
This simple change can dramatically reduce the number of objects created and stored in memory.
00:14:05.040
Another tool available for memory analysis is the middleware designed by Matt Ayman, which tracks garbage collection activity.
00:14:11.440
As you run your application, you gain real-time insights into how many requests have passed since the garbage collector last ran.
00:14:18.640
For those unfamiliar with Ruby’s garbage collection, be aware that MRI utilizes a simplistic method. When it’s time to collect garbage, execution pauses while the collector examines each object to determine if it’s still needed.
00:14:26.640
This can severely impact your application’s performance, as all execution halts until garbage collection completes.
00:14:32.840
If you are seeking to improve Ruby performance but dislike the current garbage collection method, consider exploring options with languages like C++.
00:14:39.440
In Ruby 1.9.3, a new feature allows manipulation of the garbage collector directly through its module, leading to various optimizations.
00:14:47.840
You can inquire about the garbage collector's activity or the number of objects being cleaned up, which helps assess efficiency during runtime.
00:14:55.040
Next, let's address database performance. This is a common area where most applications can improve.
00:15:01.680
One of my personal favorite tools for database optimization is the gem 'Bullet'. It's a handy tool that helps identify N+1 query issues.
00:15:08.640
In most cases, you may encounter situations where it pulls all parent objects and their children instead of querying the database efficiently.
00:15:15.920
It tracks these problems quietly in the background, sending alerts when it detects when your application is performing multiple queries that could be optimized.
00:15:25.840
Bullet identifies those occurrences, and the recommendation to remedy the situation can often be to use includes or eager_load to prevent what are essentially tiny, repeatable queries.
00:15:32.320
That being said, always analyze the recommendations Bullet provides to discern whether they indeed lead to performance improvements.
00:15:39.920
For example, when you utilize the includes method, you may end up running significantly slower queries instead of fixing the overall query performance.
00:15:48.560
These changes depend critically on the data structure, so gauge the size of your queried data, the frequency of access, and monitor your results consciously.
00:15:55.840
A common beginner mistake is misusing ActiveRecord methods where equivalent operations yield different results, such as length versus count.
00:16:05.280
Utilizing length retrieves all result objects, thus calling multiple database queries, while count applies a simple 'COUNT(*)', returning just an integer.
00:16:12.960
This simple change can significantly enhance performance, showcasing how precise adaptations can yield considerable improvements.
00:16:21.440
Additionally, you can implement the 'slow jam' gem, which monitors your log file and alerts you to slow queries.
00:16:28.960
This gem complements the insight gained from the Bullet gem by helping you find which actions are causing high latency.
00:16:35.520
While it may not explain why the queries are slow, knowing which queries run slow is the first step to addressing these issues.
00:16:42.560
The Rails 3.2 release introduced auto-explain features, which could also assist you in investigating slow queries, although it requires further debugging.
00:16:50.720
Unfortunately, some features released may need additional tweaks before fully operational.
00:16:57.520
My biggest takeaway today is the distinction between guessing and precise analysis of performance. One major mistake developers make is to neglect performance entirely.
00:17:05.840
The second mistake is to presume knowledge of what operations are efficient or inefficient without fitting it into measurable results.
00:17:12.080
Ultimately, avoid guessing—measure and know.
00:17:16.960
If you're interested in the slides, tools, and sample projects, be sure to visit the following GitHub repository for all related materials.
00:17:29.520
Our team at Jumpstart Lab offers top-notch Ruby and Rails training programs, and if you're interested in improving your company's skill set, please get in touch.
00:17:35.680
Thank you!
00:33:02.240
no
00:33:29.200
you
