Ruby on Rack: The Magic Between Request and Response

00:00:52 Waller: Hello everyone!

00:01:00 Today, we're going to be talking about Ruby on Rack. I call this talk "Ruby on Rack: The Magic Between Request and Response."

00:01:05 Before we get into that, I want to take you on a trip down memory lane back to a time when the internet was a mysterious realm filled with endless possibilities and websites loaded at the speed of an entire test suite run. Before I heard of the internet, I was just fascinated by computers. This is me in my family's computer room; we all had one. I loved playing games on the computer, and I enjoyed just messing with it and typing on the keyboard.

00:01:27 One of my favorite things to do was to play games on floppy disks. The Wheel of Fortune game was my favorite. I didn't know any of the clues—they were all for obscure movies I'd never heard of, but since it was just a floppy disk, there were obviously only so many puzzles. Once I figured out all the puzzles and completed them, I decided it was time to play other games. One game I really liked was spreadsheets, where I would balance a budget for a pretend family. It was a fun game!

00:01:45 I just wanted to be on the computer. But this was before our family got the internet in our house. Eventually, we joined the rest of America and went online. It all started in 2004 when I was a wide-eyed 12-year-old with a mission—to make my Neopet shop adorable and glittery. I remember crafting my first line of HTML in Microsoft Notepad, and that moment was amazing to me. I was taking letters and transforming them into something that appeared on the internet just by typing, and it felt magical.

00:02:12 I started discovering tons of websites made by girls not much older than I was, mostly teaching other girls how to write code. Some of this code was very specific to Neopets, and some of it was just code in general. This was one of my favorite websites at the time: cir.cuz. It was run by a 13-year-old girl who was spending lots of time making websites but might have been looking for something else. She wrote a bold declaration: 'I've lost all interest in web design and got a life, so leave me alone!' But did I get a life? Did I abandon the mesmerizing world of web design for something else?

00:02:38 Well, I’m sure the suspense is killing you, but I’m still here, talking to you about web application development today! So it turns out I did not get a life, and I am still here. I still mostly want people to leave me alone, though, so that didn't change. Apparently, I'm still on that quest for the perfect balance between my glittery Neopet shop and everything else.

00:03:00 But why am I sharing this with you today? It's not just a trip down memory lane; it's a metaphor for our journey here. Much like my quest to beautify my Neopet shop, I've always been curious about what happens behind the scenes on the web. That curiosity led me to explore the magic of web development, and today it leads us to a different kind of magic: the magic of Rack.

00:03:10 Just as I dove into the code to make my Neopets page play music automatically, with no way to easily make it stop, today we are going to delve deep into the heart of Ruby web development itself and uncover the secrets behind Rack. So, are you all ready to discover the magic between request and response? Let's embark on this adventure together!

00:03:38 This is where our journey is going to take us: this is the T-Web application, a fun playground that showcases the capabilities of Rack. Before we dive into Rack, I'm going to give you all a sneak peek of what we're going to be building. It has a straightforward interface that encapsulates the magic of web development using plain Ruby with Rack.

00:03:54 The thing that makes this app tick is its Rack-powered features that bring the application to life. We're talking about WebSockets for real-time interaction, template rendering for dynamic content, and custom middleware for flexible request processing. Each feature is something that we can accomplish with just plain Ruby using Rack. If you want to check out the application, feel free to scan the QR code on this slide.

00:04:30 As we unravel Rack, you can use this app on your device to let me know what you've learned. If you'd like to share, you can type it out in the text area, or you can just press the button that says, 'I learned something.' But before we get into that, we're going to dig deeper into what Rack is all about—what it proposes to do and what it does for us.

00:04:45 You might be wondering why Rack is even worth exploring. Rack is the unsung hero that forms the backbone of Ruby web applications. Think of it as the foundation that all of our popular web frameworks—like Sinatra, Rails, and Middleman—are built on top of.

00:05:10 This is a quote from Christian Neukirchen's blog post introducing Rack in 2007: 'Rack set out to provide a minimal API so that we can develop web applications in Ruby. It connects our web servers and Ruby web frameworks.' So we're going to zero in on that minimal API part to start. The beauty of Rack lies in its simplicity.

00:05:27 Here we have a straightforward Rack application in just a few lines of code. We define a class called MyApp, with a call method. The call method expects a few things. First is the environment object—this is an instance of an unfrozen hash with CGI-like headers. The Rack application is free to modify the environment, which includes information like the request method, the query string, the server port, as well as several Rack-specific variables like Rack session, logger, and the Rack input stream. You can see everything that the environment is comprised of in the Rack specification documentation.

00:05:56 The response for a Rack application has three parts. First is the HTTP status, which is required to be an integer greater than or equal to 100. Here, I've set it to 200, so we get that OK response. The next thing that the response provides is our headers. This is also an unfrozen hash with string keys. There are some special headers starting with Rack, which are for communicating with the server and should not be sent back to the client. The content type must not be set when the status is a 100 status, 204, or 304. The content length shouldn't exist when the status is 100, 204, or 304.

00:06:29 The last thing that we get in our response is our body. This is typically an array of string instances, a numeral that yields string instances, a proc instance, or a file-like object. The body has to respond to each or call a body that responds to each, which is known as an enumerable body, whereas a body that responds to call is going to be one of our streaming bodies. As you can see, the minimal API of Rack allows us to create web applications with incredible ease.

00:07:02 But Rack doesn't stop here. We're going to zoom in on another crucial aspect of the Rack mission: connecting with web servers. Rack acts as the bridge between your Ruby web application and the web server.

00:07:12 As we know, we have tons of Ruby web servers like WEBrick, Puma, and Unicorn. Rack allows us to run efficiently across these different server environments because it provides us with this great server interface. So we're going to take a look at this snippet and see how we can connect with different web servers. We're using Puma here; we just initialize it here with the Rack up handler and run Puma with our app. We can set it to the port that we want.

00:07:34 There are other options that we can set as well. Now we're telling Rack to run our application using the Puma server. This bin/rack up is going to kick off our config.ru file, so Rack seamlessly integrates with various web servers, allowing you to choose the server that best suits your application needs.

00:08:00 And the proof that this is actually working is that this is technically a web application. It's not very interesting, but still impressive, right? We did this with just that one method. Here we're using the WEBrick server, and we can see that it's just as easy to swap this out—replace Puma with something else. When we run our Rack up, we see here that WEBrick is what we're using as the HTTP server, starting at the bottom line on port 9292.

00:08:22 It's no different to the user, but we know that WEBrick is the web server that we're using. Now we're going to dive a little deeper into what happens behind the scenes because, like I said, Rack provides us with a web server interface that abstracts away many of the complexities with server communication. The abstraction is a key factor that makes our lives as developers simpler and more efficient, allowing us to focus on writing the web application code.

00:08:55 Here is an example of the WEBrick class in the Rack up handler namespace. The way that this interface works in Rack is that it handles those tedious tasks like initializing the server and managing the request-response cycle. At the heart of Rack is this web server interface, which defines a standard way for web servers to communicate with Rack applications.

00:09:15 We're going to take a look at how this abstraction simplifies the process. Down here, we see that Rack initializes the web server with the provided options. These options can include things like the host, the port, SSL settings, etc. Next, we see the server mount line, where Rack tells WEBrick to route all incoming requests to our Rack application. The slash path indicates that our Rack app will handle requests at the root level.

00:09:45 Finally, we're either going to yield the server to the block if it's given, or we will start the server, and at this point, our Rack application is up and running, ready to go. This process encapsulates the elegance of Rack's approach. With just a few lines of code, we configure, connect, and initiate the server, abstracting away all the complexities and allowing us to focus on what matters most: building our web application.

00:10:07 Now, we're going to zoom in on a little bit of the inner workings of handling requests in Rack, particularly through the Rack request class. This class handles parsing and provides access to the details of incoming HTTP requests. Understanding the request is very important so we know how to respond.

00:10:24 So we can gain insights by exploring the Rack request class. Before we get into that, I have a little confession: I'm a total puts debugger. When I want to know what's happening inside of the Rack request class, I go straight to the source, sprinkle some puts directly in the Rack library itself, and usually clone a version of it to set my `GEM_PATH` to that version so I can mess around and see what's going on.

00:10:53 This hands-on approach has been my secret weapon to understanding the nitty-gritty details of Rack, as well as other gems. So here are some puts statements I included. I'm going to output the environment at the Rack request initialize method and also inside of the namespace. When I kick off the server, we can see what the environment looks like at that point. There are lots of other places that we could also put these puts requests. The full environment is much longer than this, but there is a lot of information here that we can use.

00:11:27 Let's take a moment to reflect on what we've learned so far. We've seen Rack's minimal API and how we can easily create web applications with whatever server makes the most sense for us. The next thing Rack provides is that Ruby web frameworks are built on top of Rack, so understanding Rack can help you understand more about Rails, Sinatra, Middleman, or whatever you're building with.

00:11:59 We're going to explore those three frameworks today and see how they leverage Rack. Each of them has its own flavor of how they handle this. We're first going to start with Sinatra.

00:12:11 Sinatra is the micro framework, and we're going to dig into the source code to see how Sinatra kicks off its Rack application. First, we can see that Sinatra leverages Rack's request and response classes by inheriting from them. In our initialization, which is our builder here, we are actually kicking off the Rack Builder, using that to transform it into a fully-fledged application.

00:12:34 Next, we have Middleman, which is a static site generator and also uses Rack for its application initialization. First, we have essential Rack middleware for ensuring compliance with Rack specs and handling HTTP HEAD requests. Next, Middleman dynamically incorporates any additional middleware configured by the user, and then the root URL is mapped to the Middleman application, setting the foundation for routing all incoming requests.

00:12:53 Additionally, custom path mappings are accommodated, and finally, the application is returned. Now, here we have Rails. At the heart of Rails, we have this app method responsible for initializing the Rack application. Rack uses a lazy initialization strategy, ensuring that we're only building our Rack application when it's needed.

00:13:13 Next is the construction of our initial middleware stack. The default middleware method is called, setting the foundation for subsequent configurations, and then middleware configuration is next. Rails merges additional middleware configurations into the existing stack, and then finally, the Rack application is built using the configured middleware stack and the designated end point.

00:13:32 Now that we've learned about Rack's reason for existence and how it provides us with that nice interface for abstracting away all the not-fun parts of web development, the true beauty of Rack, in my opinion, is how intuitive it is to compose even more powerful applications by stacking Rack apps in middleware.

00:13:47 Let's take a closer look at what we mean by custom middleware and how we can use them to sort of puzzle piece our way to a web application with minimal dependencies. Here's a piece of custom middleware that I wrote; it's very simple and somewhat pointless. This middleware changes every URL that gets requested to say 'Fubar' instead. But this is something we can do with middleware because we initialize it with our Rack application, meaning we have access to it, and we can let middleware do its thing before passing it along down to the rest of the stack. Middleware needs a call method that takes in the env environment.

00:14:18 Middleware comes between the client and the server, hence the 'middle' in the name. This allows us to do things with the request and the response before returning to the server. We can process the request before sending it back to the server and down to the middleware until the response eventually returns to the client. Here, we process the request by just adding a custom HTTP header, and we can also process the response in a similar way.

00:14:48 We've seen the promises of Rack and what middleware is, so let's dig back into the T-Web app. These are some of the features we'll go over during the development process of this application. There are so many other things we could add to this, but we're limited on time.

00:15:15 As we transition into the practical part of this talk, let's dive into the development process. Our adventure begins with my very first commit to the application. At the heart of the app is the app class, which sets up our Rack app using Rack Builder. The run method specifies a simple lambda that responds with a 200 status code, empty content headers, and a string saying 'T.' The call method ensures our Rack application is launched, making it ready to handle incoming requests. It's a simple yet crucial piece of the puzzle.

00:15:38 Believe it or not, this tiny script is all we need to run our Rack application! In our config.ru file, we require our T-Web application, which uses the run method to pass in our application, and we can use the bin/rack up command to kick things off. Here is our application, and I think we're done looking good!

00:16:02 After getting my initial Rack up, I thought it was time to bring in some style. I added Slim templates for rendering. My choice in this was primarily motivated by how clean it would look in screenshots. Now, in our Rack application, we still only return that one response, but now we're doing more than just returning a static string. We've replaced the 'T' string with a call to a render template method.

00:16:23 In our render template method, we're finding the template in the views directory by the name parameter and then running that through the Slim template render implementation. This will spit out a string of HTML with any dynamic content interpolated. Our config.ru file has also changed slightly; I've introduced the Rack Static middleware to handle our static assets. I've created a public assets directory for storing the CSS that I created for this app.

00:16:50 Please don't view source if you have a deep fondness for well-written semantic or idiomatic CSS—I'm not a front-end developer, and I did my best! Here's our new index.slim template, which is basically just a shim for our landing page so it looks right, but it doesn't have any actual practical functionality yet.

00:17:05 This is where our Rack Static middleware piece comes into play. Rack will know how to handle our style sheets by looking in the public directory. At this point, any URL we go to will return this page because it is the only response the application knows how to return. It will always show a 200 HTTP status code and return the index view.

00:17:30 To add a new route, we just need to map it in our T-Web routes. We're using the same render template method, sending the template file name as the argument. Now we've got shims for both of our pages, and we're ready to start adding some functionality. The updated config.ru file is also where we're going to go when we want to add persistence.

00:17:57 I'm doing this with SQL's ORM as well as PostgreSQL. For anyone unfamiliar, ORM stands for Object Relational Mapper—it functions like Active Record. The SQL connect line establishes a connection to our database, seamlessly integrating into our application and allowing us to have persistence.

00:18:15 Now, our actual app.rb has changed quite a lot. Here's the complete file after introducing our new routes for posting our learnings, as well as now rendering a 404 page for unknown routes. We're going to zoom in on the new things we've added. We established this vital link between our application and the database by allowing us to pass our database connection during initialization from our config.ru file.

00:18:47 As we move forward, we'll see how we use this database connection. Next, we go into our call method, which is our navigator guiding the app through all of the incoming requests. In the call method, we have a routing mechanism. As we dissect the call method, we can see how it's channeling requests to their respective destinations based on the request path and method.

00:19:11 Here we are at the T-Web path, and if the request method is GET, we're displaying the T-Web. We retrieve the T-Web and set them to an empty array before rendering the view, passing that T-Web array as a local variable. This allows us to access the inner Slim template so we can have dynamic content. Here, we are using the SQL Object Relational Mapper, which allows us to abstract away that low-level SQL stuff and have this nice API.

00:19:36 We're grabbing the records from the learnings table, excluding those where the content is nil, and then we're ordering them by ID. Now, our creation route: if the request method is POST on the T-Web route, we will run through this method. We grab content parameters from the request and insert them into the learnings table in our database. After that, we re-render the index page, which allows the new learning to be displayed on the page.

00:19:58 Our index.slim view has an added form to post to this route with the content parameter. As I mentioned, we connect to our database in our config.ru file. I didn't include the SQL rake task I added for database migrations, but the process for adding database migrations in Rack with the rake task for SQL is outlined well in their README. If you're interested in learning how to integrate that into a plain Ruby Rack application, definitely check that out.

00:20:21 Now we have persistence for our learnings, but we also want to track that counter on the front page. It didn't seem right to me to store it in the database, as it's not tied to another record. So I decided to introduce persistence in a different way by creating a text file that stores directly on our server.

00:20:38 In our app.rb, we are now setting our counter, which we can use as necessary in the rest of our application. We are now setting this to read from our til_count.txt file and then changing it into an integer for us. We've added a new branch for our root path; if the request method is POST, like we saw with our learnings route, we will call the increment counter method.

00:20:58 The increment counter method is quite naive—we're just incrementing the counter by one, writing it to the file, and then returning the index response. I've also introduced a new static asset, the counter.js file, and added an ID to our button so we can easily access it in our newly created JavaScript. The counter.js file will send the POST request directly to the server, then change the counter on the index page in place, giving it the feeling of asynchronous loading.

00:21:17 Let's spin up the server and take a look at our application in its current state. Now, anyone see any issues? I'm clicking every time, and this is what's happening here. I think it might be time for a rate limiter; after all, nobody can learn that fast!

00:21:45 So as our application grows, so does our need to be responsible with its use. We are introducing rate limiter middleware, which ensures that resources are used wisely and prevents any single user from overwhelming the application. Our rate limiter starts off by defining what the rate limit in the time window is. I've decided maybe 10 learnings per minute is appropriate—every six seconds seems like a pretty high bar, but I don't think we could be learning at a faster rate than that.

00:22:05 Here, storage will track the requests per session, and every time we get a request, the rate limiter checks if the session has exceeded its allotted requests within the specified time window. We have this rate limit exceeded method that assesses whether the session has gone past its request limit.

00:22:28 First, we check if the request is a POST method and if the session has recorded any requests already. We calculate the current time and retrieve the count of requests with the timestamp of the oldest request for the session. Next, we ensure the time elapsed since the oldest request does not exceed the specified time window, and then we assess whether the request count surpasses the set rate limit.

00:22:51 As the rate limit exceeded method does its job, it evaluates each request, ensuring that no one session monopolizes our application. But what happens when a session reaches its limit? This is where the update rate limit method comes into play.

00:23:12 You can think of it as a log—it records each request and initializes the session's request history if it doesn't exist yet. The method timestamps the current request and adds it to the session's history, and if the request count exceeds the rate limit, it prunes the oldest request, maintaining a record of the most recent ones. We've also added some new Rack middleware; the rate limiter is our custom Rack middleware.

00:23:34 But I'm also using this Rack session cookie provided by Rack—it's cookie-based storage, which we use for our user sessions. This needs to stack on top of our rate limiter middleware since we need access to it. Let’s check it out now!

00:23:51 Let's take a look at what's actually happening behind the scenes. If we check the console and look at the network tab, we can also check the console tab to see that we are getting rate limited. It's just not displaying due to our JavaScript file augmenting it in place.

00:24:08 Here’s what you would see in the console as you run your server: you would see the 200 requests come through, and then you would just get 429s after that. I decided to create a cool down on the button of six seconds; you won’t be able to press it in the meantime. So now, when you press it, I also added confetti—just for you—but you won’t be able to press it again until the six seconds have passed.

00:24:29 Our application has grown again because now we're introducing WebSockets! Just to backtrack for a second, I actually gave this talk for the first time ten years ago when I was a software apprentice. I wanted to understand how Rails worked and thought I would walk through the application, putting debug statements everywhere. It was too much! Rails is so huge, but what I found most interesting was everything happening in Rack.

00:24:49 That's how this talk originally came to be—just me digging into Rack to understand it. Now, I'm revisiting it ten years later, and there are many cool new things. I had to add WebSockets to the application, which allows us to get instant updates across all of our clients through real-time communication. It's a super cool feature that is not complex to implement into a plain Rack application.

00:25:08 We're using the Faye WebSocket library to start. We initialize our connections to an empty array, and then inside of our call method, we check if the WebSocket exists. If it does, we'll set up listeners to handle events for opening the WebSocket, receiving messages, and closing the connection. When a learning is created, we also call the notify til update method after inserting it into the learnings in our database.

00:25:43 The notify til method is the method that broadcasts to the WebSocket to all connected clients, ensuring that everyone stays in the loop in real time. In our Slim view, nested under a JavaScript section, we create a WebSocket at our desired path and then set up our listeners. We have listeners for when it opens, and when the button is clicked, we grab the content and send it through the socket. When a message comes through, we’ll increment the counter, reset the form, and add the new learning into our list.

00:26:01 Let's try adding something we learned. Keep in mind, this doesn't really prove that WebSockets are working because we're currently just showing the one screen. We can see that when I have two browser windows open and put my learning in one side, it updates both instantaneously.

00:26:20 As for where to go next, there are so many options to extend this project and delve into Rack. We didn’t even scratch the surface of the possibilities available. Just sticking with the standard Rack Middleware library provides us with many important pieces for creating robust and powerful web applications.

00:26:41 I highly encourage everyone to read the Rack docs. Instead of reaching for a full-fledged framework, it might make more sense to build something totally custom—you should try it; it's fun! But even if you never want to build your own Rack application, just knowing about Rack unlocks pieces of other frameworks that might have been fuzzy or that you never really thought about.

00:27:02 One custom middleware I wanted to build but didn’t get to is a custom routing middleware so that we could have a routes.rb file, which would eliminate the need to change and add a new case statement for routes. That would be my next step, I think—to make it more flexible. There’s a lot of opportunity for cool custom middleware here.

00:27:20 I'm actually toying with the idea of adding this routing layer. I will likely push up some pull requests to the repo for this, so if anyone's interested in playing around and contributing, you can fork it and clone it—it's on my GitHub.

00:27:34 Of course, I didn’t include a link on the slide, but I can post it in the Slack if you'd like. This dive into Rack could lead you to enjoy our Test Double Dispatch, because at Test Double, we are always doing cool things with projects, open source, and we want to share what we are doing with you. If you sign up for our newsletter, you will get emails directly to your inbox with cool updates.