Summarized using AI

ActiveSupport::Notifications and Live Status Pages

Matthew Delves • June 27, 2014 • Singapore • Talk

In this presentation at the Red Dot Ruby Conference 2014, Matthew Delves explores the use of ActiveSupport::Notifications and how they can be utilized to create effective live status pages that provide real-time insights into a website's performance.

The discussion centers around several key points:

  • Role of Operations (Ops): Matthew humorously contrasts the stereotypical images of operations staff—depicting them as both remote and grumpy—but emphasizes that ops can be approachable and are crucial in ensuring the smooth operation of code.

  • ActiveSupport::Notifications: This is the backbone for gathering insights. Matthew explains that it consists of instrumentals (which offer notifications on various actions such as controller processing and SQL interactions) and subscribers (which listen for these notifications and allow for actionable insights).

  • Asking the Right Questions: He emphasizes the importance of formulating clear, measurable questions to gather useful data, highlighting that a good question leads to better insights.

  • Custom Instrumentation: Delves advises that while creating custom notifications can be powerful, it’s important to use them properly (primarily for statistical purposes instead of altering application state).

  • Building a Live Status Page: Matthew outlines the components needed for a live status page, which include:

    • Notifications to track relevant metrics.
    • A time series database to handle and visualize data over time.
    • A graphing library to present metrics in visually appealing ways, such as D3.js.
    • ActionCable, which allows the server to push real-time updates to the browser without constant polling.
    • Rails Engines to structure the code and facilitate reuse across applications.
  • Importance of Data Management: He discusses selecting the right type of database (like InfluxDB or Graphite) to effectively store and query time series data, as well as considering future uses of the data for analytics and metrics collection.

  • Single Responsibility Principle: The presentation underscores the benefit of using Rails Engines for live status pages to maintain a clean architecture and reusable code.

The takeaway from Delves' presentation is the integration of ActiveSupport::Notifications in the development of live status pages, crucial for providing stakeholders with the real-time performance of their applications, thereby improving communication between development and operations teams. The session wraps up with an invitation for questions, ensuring an interactive environment for the audience at the conference.

ActiveSupport::Notifications and Live Status Pages
Matthew Delves • June 27, 2014 • Singapore • Talk

An overview of ActiveSupport::Notifications and how they form the backbone of a live status page to tell you exactly what is going on with your site.

Help us caption & translate this video!

http://amara.org/v/FGYb/

Red Dot Ruby Conference 2014

00:00:19.230 yep cool Radio my cloaking just waiting
00:00:23.019 on a clicker which will eventually get
00:00:25.300 but first a public service announcement
00:00:28.779 I and matt & M ops guy which means I
00:00:34.300 look like this guy and I live in the
00:00:36.730 server room but that is not something to
00:00:39.789 be feared now the other opinion of ops
00:00:45.429 people it's this guy mr. grumpy now why
00:00:51.670 ops people consider to be grumpy well
00:00:55.379 the truth is we care a lot about your
00:00:58.539 code and we want to make it run well and
00:01:01.170 the best way to make your code run well
00:01:04.660 is to constantly talk with you guys and
00:01:07.360 be in communication now we have a bit of
00:01:12.340 a bad reputation in that we're
00:01:14.740 considered untouchable and just
00:01:17.530 unapproachable there is a good solution
00:01:20.500 to this and it is beer oops people like
00:01:26.290 beer we can be very easily bribed with
00:01:29.380 beer and if you think that we
00:01:33.810 unapproachable and friendly just come up
00:01:36.880 and say hey need your advice here's a
00:01:40.090 beer let's just have a chat so on to the
00:01:46.300 doc active support notifications and
00:01:49.000 live status pages go I'll switch back to
00:01:58.690 the pulpit
00:02:02.750 okay cool so to begin with let me tell
00:02:08.190 you a story you're gonna like my slides
00:02:12.620 it is a monday morning and you're in the
00:02:16.290 office early you decided yep I'll just
00:02:20.489 get in there start the week off well get
00:02:23.910 through all of your emails and you
00:02:25.830 manage to do so you're on top of
00:02:28.440 everything and it's awesome sadly though
00:02:34.170 things are about to conspire against you
00:02:37.459 because of this guy the boss is there
00:02:40.980 and he just comes up to your desk and
00:02:44.069 goes well I'll showing this side to
00:02:48.269 investors and it seems that all's
00:02:52.200 running a bit slow so i'm going to need
00:02:55.079 you to stay back late all this week and
00:02:57.950 get it fixed this is how you respond
00:03:05.930 pretty much no information as to what he
00:03:09.540 was trying to do and no leads whatsoever
00:03:13.579 so how do you get this problem solved
00:03:18.269 you can start off by looking at things
00:03:20.700 like New Relic skylight numerous other
00:03:24.989 things that will actually give you some
00:03:28.680 insight but you think yeah there must be
00:03:32.310 a better way to get dissolved now
00:03:36.260 serving off we need to take a step back
00:03:40.019 from all of the technology and look at
00:03:42.690 how we're going to actually get the data
00:03:46.290 out that we need and what we need to do
00:03:49.319 as far as an approach so hands up if
00:03:53.700 this is your response to doing any kind
00:03:56.790 of data query don't be shy to like you
00:04:01.500 just love doing this stuff I've got one
00:04:03.389 guy down the front who does that's
00:04:05.190 awesome yeah for those that don't get
00:04:09.989 the reference this is data and he's
00:04:12.389 doing a search for life forms it is
00:04:14.669 awesome
00:04:15.709 so how do you actually search for good
00:04:20.549 data you need to ask good questions and
00:04:25.110 what makes a good question it is
00:04:28.380 something that is simple measurable
00:04:31.229 achievable realistic and timely I've
00:04:35.510 completely plagiarized this from
00:04:38.400 something in a different context but it
00:04:40.620 still applies the questions that you
00:04:43.320 need to ask have to actually result in
00:04:47.280 you getting the right data out otherwise
00:04:51.419 it is just completely useless now when
00:04:54.990 you start off you'll often get many
00:04:57.990 situations whereby you just want to do
00:05:00.930 this just go screw it we're starting
00:05:05.100 over that's it so active support
00:05:10.760 notifications now as with anything there
00:05:15.539 are sort of two sides to it the first is
00:05:18.889 instrumentals and the second is
00:05:21.979 subscribers so let's have a look at
00:05:24.539 instrumentals now in rails you've
00:05:28.349 already got a lot of stuff which is
00:05:31.099 instrumented for you by default you've
00:05:34.919 got a notification of when a controller
00:05:38.160 starts processing something when it
00:05:41.130 processes an action renders are partial
00:05:44.520 it does some stuff with SQL and there's
00:05:48.060 a lot more as well now from this you
00:05:51.120 should get a good idea of what is
00:05:53.370 already possible with notifications and
00:05:57.900 how best to use them now if you want to
00:06:03.150 know the full list the rails api docs
00:06:05.669 have them the rails guides as well give
00:06:09.360 a really good introduction into how to
00:06:11.789 use this stuff but you can also do this
00:06:15.060 really cool thing of custom
00:06:17.780 instrumentation so if you wrap your code
00:06:22.349 in something like this you are able to
00:06:25.590 see and instrument
00:06:28.990 what that particular bit of code is
00:06:31.750 doing now this is fairly powerful in
00:06:37.090 that you've got a hash there which is
00:06:40.420 your search and params that gets passed
00:06:44.230 through to your subscriber and you can
00:06:48.340 pass anything through to that now this
00:06:52.180 though comes with a big fat warning
00:06:54.900 because there be dragons you can do a
00:06:59.860 lot of stuff with notifications and some
00:07:04.870 people think that it's a very simple way
00:07:07.810 of doing a publish-subscribe
00:07:11.370 notification system in that they will
00:07:14.050 put a custom instrumentation call around
00:07:19.720 a bit of code and then try and use that
00:07:24.850 to say update the state of a model or
00:07:27.700 something rather weird like that this
00:07:31.480 isn't the best thing to do with
00:07:33.160 instrumentation because it's designed to
00:07:35.710 be gathering statistics about what's
00:07:38.620 going on so let's have a look at
00:07:42.390 subscribers this here is a subscriber
00:07:46.650 you put this in an initializer and it
00:07:51.280 will listen in this case for anything
00:07:54.630 for the notification that gets sent when
00:07:57.730 there is an active record SQL call it
00:08:01.810 will be passed the name you get a start
00:08:04.900 and finish time you get a unique ID and
00:08:08.980 you get a payload and within that you
00:08:13.030 can do some rather awesome Ruby's stuff
00:08:17.760 so it's a basic overview let's have a
00:08:22.060 look at what it looks like in practice
00:08:25.620 so by practice I mean the other part of
00:08:28.990 the talk title which is doing a live
00:08:31.720 status page first thing you need is you
00:08:34.900 need your notifications you also need a
00:08:38.320 time series data base because what we're
00:08:41.620 doing is
00:08:42.750 making metrics over time having
00:08:46.080 something like a time series data base
00:08:48.840 in there is pretty good a graphing
00:08:52.410 library because everyone likes pretty
00:08:54.480 graphs action control alive is really
00:08:58.920 useful because you don't want to be
00:09:00.840 constantly polling your server for new
00:09:04.170 data you want to be told has it updates
00:09:06.990 and having it all wrapped in a nice
00:09:10.260 rails engine is also pretty good so
00:09:15.260 something like this is what will fit in
00:09:18.420 your initializer you've got a stack of
00:09:21.920 notification subscriptions all doing the
00:09:25.320 one thing now what you can also do is
00:09:30.260 regex matching so something like this
00:09:33.930 will listen for anything that matches
00:09:36.360 that particular regex now this is pretty
00:09:40.200 powerful if you want to make sure that
00:09:42.810 your code isn't copied and pasted
00:09:45.420 multiple times it's just a great way of
00:09:48.589 doing using the dry principle pretty
00:09:52.680 awesome the time series data base now
00:09:56.130 what are some examples of a time series
00:09:58.740 data base got graphite and carbon and
00:10:02.930 which is developed by etsy that's pretty
00:10:06.300 good written in Python but we won't hold
00:10:09.150 it against them and it's a reasonable
00:10:13.650 time series data base hobbes people tend
00:10:16.080 to use it fairly regularly for a lot of
00:10:18.870 their metrics got something like in flux
00:10:22.410 DB which I tend to like again the time
00:10:25.770 series data base but has some SQL like
00:10:30.560 functionality to it which really helped
00:10:33.180 spur query tempo DB can use Redis but
00:10:38.910 you end up with the problem of it just
00:10:42.030 being an ever-growing Redis store you
00:10:46.230 can also just use a postgres which is
00:10:49.140 pretty awesome because it's pretty
00:10:51.000 powerful but it's a lot heavier than
00:10:54.810 what you would really want
00:10:57.890 you must use the right debate when doing
00:11:02.550 anything with data with statistics you
00:11:07.470 need to not only think about what the
00:11:10.290 data is going in but how it is that you
00:11:13.440 will use it so if say you're sticking
00:11:18.150 JSON objects everywhere it becomes
00:11:20.880 fairly difficult to do things like
00:11:23.220 averages max men and stuff like that
00:11:27.150 which a straight value store will will
00:11:33.960 allow you for so make sure you're going
00:11:37.080 to be asking the right questions think
00:11:40.140 about how you will use the data don't
00:11:43.290 only go this is how I'm immediately
00:11:46.410 going to use the data but rather think
00:11:49.080 this is how I want to use the data down
00:11:52.170 the track reason i say that is over a
00:11:55.590 couple of months you'll get a large
00:11:57.870 amount of data and you then go well okay
00:12:01.410 let's try and do some funky stuff with
00:12:04.430 how this data is used if you're storing
00:12:07.680 it in the wrong place if you're storing
00:12:10.170 the wrong things then that data is
00:12:13.560 pretty much worthless for you and you'll
00:12:15.510 need to start again there is no
00:12:19.110 one-size-fits-all you need to think
00:12:23.280 seriously about how it is you're going
00:12:25.830 to approach the problem in my particular
00:12:29.520 case I chose influx d be it is a simple
00:12:34.470 approach there's a really good really
00:12:37.080 ruby gem and you've even got a rails gem
00:12:40.650 which handles some notification type
00:12:43.740 stuff as well it's got a query language
00:12:47.130 so you can do things like count average
00:12:51.360 men max median mode whatever you want
00:12:54.780 and storing it is as simple as this
00:12:59.070 digit create a data hash which has a
00:13:02.430 value and a time and you then just write
00:13:05.610 it to your data database
00:13:09.710 retrieving the data is where it becomes
00:13:12.060 really cool because you can use select
00:13:15.300 queries so get all of the values between
00:13:18.120 a particular time you can limit it to
00:13:21.060 some results as well you can do counts
00:13:24.900 over time you can do a range of really
00:13:28.590 awesome things graphing library this is
00:13:34.770 all JavaScript and front-end stuff I do
00:13:39.170 quite often badmouth JavaScript but it
00:13:42.690 has its uses in this case there's is
00:13:46.350 really amazing library called d3 and
00:13:50.000 there's also another library called
00:13:52.710 ritual which sits on top of d3 and makes
00:13:57.140 the calls to d3 a lot easier so it's a
00:14:01.950 convenience wrapper this is what it
00:14:05.160 looks like to create a graph just say
00:14:08.190 new graph you provided with the Dom
00:14:10.350 element you set up the series so you can
00:14:14.070 do multiple series in the one graph and
00:14:16.500 then in call render this is a very cut
00:14:20.460 down and reduced bit of code but it
00:14:25.140 gives the idea of how simple it is
00:14:28.790 action control alive so first heard
00:14:33.090 about this from Erin a couple of years
00:14:35.610 ago and it is really amazing so thank
00:14:38.310 you erin it is a way of doing service
00:14:41.340 and events whereby the server will send
00:14:46.470 data through to your browser whenever it
00:14:50.280 has updated information or whatever you
00:14:55.110 tell it to and there's no need for
00:14:57.960 polling so you don't constantly have the
00:15:01.200 browser page refreshing itself for an
00:15:03.630 AJAX call or something going off rails
00:15:08.430 engines so everyone likes being able to
00:15:14.160 sort of put their code into a nice
00:15:17.010 convenient package and rails engines
00:15:20.130 allows this so
00:15:22.830 the in the example of a live status page
00:15:26.180 using a rails engine makes a lot of
00:15:28.860 sense you take your code and you put it
00:15:32.760 as a self-contained unit and that is
00:15:35.850 really good so it allows for reusability
00:15:38.840 what you write for one app can then be
00:15:42.060 taken and used again and again and again
00:15:44.880 if it's all done right easy integration
00:15:48.660 in that you just provide it as a gem and
00:15:52.940 if you want to change the default mount
00:15:55.710 point you can do that it's a single
00:15:58.890 responsibility thing so the gem which is
00:16:03.600 a rails engine has a single
00:16:06.120 responsibility which is providing your
00:16:09.330 large Status page so what it looks like
00:16:13.460 it's all up on github but it's got a
00:16:18.570 semantic version of zero point zero
00:16:20.490 point one which means have a good look
00:16:23.700 at it to see how it's done but don't
00:16:27.330 expect too much from it because I'm
00:16:30.180 still working on it yeah not quite I'm
00:16:34.620 Matt dolls that's me pretty much
00:16:38.580 everywhere on the internet I work for a
00:16:43.890 group called ray interactive and any
00:16:48.210 questions well if not thank you
00:16:57.910 you
Explore all talks recorded at Red Dot Ruby Conference 2014
+20