00:00:07.000
In the next 30 minutes, we're going to implement an RFC 2324 server together. Now, that may sound like a fairly dry topic, but RFC 2324 is all about coffee. Coffee is a liquid; liquids are wet, and therefore, this talk is certified not dry. Just in case there are any chemists in the audience, as I was procrastinating while preparing this talk, I started thinking: are all liquids actually wet? What makes something wet, anyway?
00:00:14.360
I consulted multiple experts, and I can tell you now that apparently not all liquids are wet. Both mercury and a certain alloy of gallium and indium feel dry to the touch of human skin. Who knew? And that leads perfectly into a content warning: this talk will contain bad puns, pointless tangents, and pedantry. Or was that ped? I've put it in different ways; it depends on who you ask. Anyway, you've been warned! So, let's talk about hypertext coffee pot control protocol.
00:00:31.480
That name is a bit of a mouthful; even the abbreviation is quite long. So for this talk, I will simplify it by referring to it as 'CP Squ.' CP Squ was published on the 1st of April, 1998, which is a bit of a strange choice. There have been many products announced on the 1st of April that haven't panned out.
00:00:40.480
For example, I'm still waiting for powdered water, edible computer discs, and wipers for BMW badges—not that I have a BMW, but if I did, I’d want wipers for it. Other examples include the chippy music player and left-handed laptops. All of these things were announced on the 1st of April and have not been delivered, which is very disappointing. However, I am pleased to say that CP Squ is still around today, and I think that makes it just as important as some of these other survivors.
00:01:01.600
For instance, Apple Computer was founded on the 1st of April, long ago, and while they are doomed, they're still limping along. Linux and its various distributions are also significant survivors. Next year, it will be big on the desktop too, and of course, we have Gmail, which, with apologies to Winston Churchill, is the worst form of email, except for all the others. Let's not forget CP Squ, which is the subject of international conference talks. Very important!
00:01:18.000
Now, enough prattling; let's get on with RFC 2324. CP Squ is actually an extension to HTTP. So first, we'll go through some background on HTTP to ensure we all understand it. HTTP is used by both browsers and native apps on phones as a protocol for communication. There are other protocols that also extend it, like WebDAV.
00:01:28.480
It's a stateless protocol, meaning that the communication happens through a request-response cycle. There are many different types of requests, and I’ll let you look them up on MDN. For the purpose of this quick overview, we will focus on GET and POST, two of the most common methods.
00:01:43.360
As a general rule, you use GET to fetch web pages or download funny GIFs, while POST is used to submit data, like purchasing a rubber chicken purse or uploading a file. When you visit a web page, your browser likely makes a whole bunch of requests—not only for the main HTML page but also for images, stylesheets, scripts, and so on. Additionally, those scripts can make further requests.
00:02:03.120
Most of you probably know this already, but you can view all the requests being made using the developer tools in your preferred browser. For instance, in Safari, you can check the 'Timelines' tab to see all requests that were made for a particular page, including their method, the scheme used, the status code from the web server, and whether or not it was cached.
00:02:17.360
Here, you can see that most of the requests are GET requests, along with a single POST request to Google Analytics. To see even more details about a particular request, head over to the 'Network' tab. Choose one of the requests from the left, and you can see more information. For example, if you want to see the headers, you can click on the 'Headers' tab. Here, you can view the headers from the request that was sent, as well as the headers from the response that came back.
00:02:33.000
But what are headers? No, not that sort of header! Headers are just metadata about either a request or a response, so they are essentially data about data. When we request an image from a web server, the headers might contain information that seems complicated. However, if you split it across separate lines and order it by Q value, which is just a fancy way of saying preference, you can start to see that the most preferred format is listed first.
00:02:47.679
If that fails, we will accept any other image or video, and failing that, give us whatever you like. The response from the server will tell you what it has provided, so you know which one to decode. For example, if it's an image, you might see a content type of image/png. I did warn you there might be tangents, and it’s time for a quick tangent; if you're a web developer, you sometimes have complex forms that require testing frequently.
00:03:05.040
Instead of filling out the form repeatedly after each change to your app, you can fill it out once and save the request as a cURL command. You can then paste that into your console whenever you make changes or even include it in various testing scripts. One gotcha I discovered when I was testing this out is that if the response comes back compressed, it will present a bunch of binary output that your terminal may struggle to display. There are different ways to get around this.
00:03:21.200
For instance, you can pipe it to Gzip, or add a compression flag at the end, and cURL will handle it for you. Alternatively, you can just remove that Accept-Encoding header that we discussed earlier, so here we're getting rid of gzip, deflate, and br compression. Alright, end of that tangent! Now, something all HTTP requests have in common is that they need a destination. You need to specify where they're going.
00:03:40.440
The destination of a request is specified by the URI, also known as the URL. Although technically, there is overlap, they are different, but for simplicity, I’ll just refer to it as URL for the remainder of the talk. The URL contains different portions: the scheme at the start, which we've mentioned earlier, serves all sorts of various protocols. The colon acts as a separator, and the double slashes are a network path designator, followed by the domain name, which can also be an IP address, and finally, the absolute path to the resource on the server you are requesting.
00:03:58.160
Once the HTTP server receives the request, it sends a response that can fall into one of five different categories, with variations within each category. The server will return a status code and a status text to provide more information. For instance, if you get a 200 OK message, it starts with two, indicating it is in the success bucket. A 301 move permanently starts with a three, indicating it is a redirect. A 404 not found is a client error, because in that case, the client can do something about it—like change the URL and send the request again, at which point it might work.
00:04:17.280
Last, we have a 503 service unavailable as another example. If our web server can’t communicate with our Ruby app server, it might return that, and the client can’t do anything about it; it has to go reboot it or fix whatever is broken. So, that's HTTP. Now, let’s quickly chat about how CP Squ is different from HTTP, other than the obvious changes in hardware.
00:04:35.200
The first difference is that CP Squ adds two different request methods. Specifically, it adds 'Brew,' which is used to tell when the coffee pot should start or stop brewing, and 'When,' which tells it when to stop pouring milk. Great dad joke there! If the coffee pot supports a feature, another change is for the scheme; it uses coffee instead of HTTP or HTTPS.
00:04:51.760
And that leads me to another tangent! The web can sometimes be a bit US-centric, but coffee is international. The developers who designed CP Squ actually came up with a bunch of different translations you can use. For example, the word for coffee in Finland is 'kafe'—excuse my pronunciation. These two URLs are actually equivalent. And since we're in Thailand, we should probably use the Thai word for coffee, which is written over there. Unfortunately, URLs aren't really good at Unicode, unlike Ruby, so you must percent encode it.
00:05:07.760
For the purpose of this talk, I'm going to stick with the 'kafe' pronunciation and spelling. A little tangent ended there! So, what else is different with CP Squ? It has some header changes. For instance, the content type should be 'message/coffee.' The body of the message needs to state 'coffee message body start' or 'coffee message body stop.' The last change is in the responses; there are two new errors, specifically in the 400 series.
00:05:22.800
The first is '406 Not Acceptable,' which gets reused. So if you request cream, syrup, or sugar or something else the coffee pot can't provide, it is supposed to return 'Not Acceptable.' If you request coffee from a teapot, however, you should get the infamous error '418 I'm a Teapot.' According to the specification, the resulting entity body may be short and stout.
00:05:38.960
Alright, so that's the requirements for CP Squ. Now it’s time to implement it! But that begs the question: how are we actually going to implement it? It's fairly straightforward, but should we build our own web server? Probably not. There's already a bunch of great options out there, and building our own web server means we can't use this with other existing applications.
00:06:00.000
For example, if Shopify wants to integrate a coffee brewing service for their patrons, they wouldn’t be able to do that without changing web servers, or maybe Stripe wants to connect CP Squ with one of their internal web services to improve caffeination and employee productivity. That won't work unless they change web servers just for that. By the way, if anyone from Shopify or Stripe is in the audience or watching online, I've got some brilliant ideas—give me a call!
00:06:17.480
Clearly, it makes more sense to build it into one of the web frameworks. The next question is, which one? Here are the four most popular, according to GitHub stars, but I have to say that once I considered integrating CP Squ into each of these, it doesn’t look good. Last year, Hanami joined the anti-liquid crew while Rails spat out coffee five years ago. Sinatra has had a recipes repo since 2011, and in all that time, no one has added a recipe for coffee. I mean, how hard is the recipe for coffee?
00:06:35.320
This means we're relying on Grape to implement it. I’m sure the maintainers of Grape are nice people, but let’s be honest—I figured there was no point even trying. I mean, why press grape when all you're going to get is a little wine? I'll let you think about that one. Thankfully, in Ruby, there is a way to integrate with almost any web server and any web framework, and that way is Rack.
00:06:50.280
Rack is a specification that allows any compatible web framework to run with any compatible server. It's basically a bridge between both systems. The specification also allows us to run additional software in the middle. Unfortunately, that middleware doesn't have a cool name like 'Ultra Connector' or 'Micro Bridge'—the term is simply 'middleware.' Pretty obvious! I noticed that naming stuff can be hard. Personally, I think that if you find naming stuff hard, you’re asking the wrong people. If you want something named, ask an Aussie!
00:07:08.120
To prove that, I'm going to get some help from the audience, and for that, I need to find an Aussie. There’s one surefire way to find an Aussie in any crowd—Aussie! Oi! I heard a few shouts in the audience. If you heard 'Oi' near you, please point to the person who said it. There’s one over here! Anyone who's not a speaker, please come on up!
00:07:24.720
Alright, unmute that! We're going to play a game called 'Name Stuff.' This is going to prove that Aussies are great at naming stuff. Hello, testing one-two! I’m going to show you some slides; you can either look at them here or up there. You've got to come up with a name; I'll also give you a verbal description. Where's the cheat sheet? Alright, so there’s a harbor in Sydney—what's it called? The Sydney Harbor! Well done!
00:07:37.360
There’s a bridge that goes over Sydney Harbor—what's that called? The Sydney Harbor Bridge! Good name! The bridge got a bit crowded, so we built a tunnel underneath—what do we call the tunnel under Sydney Harbor? The Sydney Harbor Tunnel! Well done! Moving on, I’ve got a spider with a red back—what could I call that? A red back spider! Nice. What about a snake that's brown? The brown snake!
00:07:57.920
Alright, I think some people are getting uncomfortable with the venomous things, so let’s move on to some tourist attractions. In Coffs Harbor, there’s a big banana—what would we call it? The Big Banana! Genius. This one’s a bit rare: in Adaminy, there’s a giant trout—what do we call this giant trout? The Giant Trout! Big Trout! Nice!
00:08:08.480
And when the snow melts, it feeds a river—what would we call that river? The Snowy River! Oh, and there’s a famous poem about a man that comes from this region by Banjo Patterson—what’s the poem called? The Man from Snowy River! Genius! 10 out of 10! As a thank you, I’ve got these genuinely Aussie-made lollies; feel free to share them at the after-party tonight.
00:08:21.760
So that’s how naming is done Aussie-style! It’s really not hard at all. Okay, where were we? Oh yes! Rack middleware! Great name—it says it right there on the tin! So, Rack isn’t just one of those specifications that someone made up for an April Fool's gag; it’s an actual specification that’s widely used.
00:08:39.000
Let's take a quick look at how it's used in two popular web frameworks. First, we've got Rails. When you start a new Rails app, you can check all the middleware included by running the command `rails middleware`. So, I did that in Rails 7, and there were 28 middleware objects, rack middleware objects, included by default. Rails has a reputation for being 'batteries included,' so you kind of expect that.
00:08:56.720
On the other end of the extreme, you’ve got Hanami, which has a reputation for letting you pick and choose what you want to include. Hanami even has a similar command: `hanami middleware`, and with a brand new project, it includes some Rack middleware by default—just the one, but still, it shows that Rack is used throughout Ruby.
00:09:14.560
So, if we're going to build CP Squ, we should do it in Rack. But how do we actually go about building something in Rack? The Rack specification is actually fairly straightforward. It requires a Ruby object, not a class, that responds to 'call.' It takes exactly one argument, the environment, and returns a non-frozen array of exactly three values: the status, the headers, and the body.
00:09:31.880
What would that look like in practice? Here’s a diagram showing what it might look like if we didn’t use any middleware; we're just using Rack. We’ve got the web browser, a web server running Puma, and a Ruby app running Sinatra. So when the client sends a GET request, Puma takes that and puts it in the environment, and then it calls 'app.call' to the Sinatra app.
00:09:49.200
Sinatra processes the response and then returns that three-part array to—status code, headers, and body. Then Puma turns that into HTML and sends it back to the client. Pretty straightforward! The clever part of this design is that it allows you to chain applications together.
00:10:04.560
So you can have zero middleware, like we have here, one like Hanami starts off with, or you can have 28 like Rails starts with. If you want, you can even try to add a thousand of your own! Let's do that ourselves. First, we’re going to take that same middleware and modify it slightly. The first thing we need to do is use the initializer to tell it what the next link in the chain is.
00:10:20.880
So, whatever gets passed to the initializer will be the next link, and then whatever we receive from the environment, we can just pass that on to the next link. Sounds simple! Let’s modify that diagram: we now have Rack in the middle. So, Puma still does 'app.call,' but this time it goes to our nullable, do-nothing middleware, which just sends that straight back to Puma.
00:10:38.800
So, what’s the point? Well, it shows the power of Rack middleware—it allows you to make changes. You can make changes on the way to the app server, and you can also make changes on the way back to the client.
00:10:57.360
In my first tangent, I mentioned we could modify our cURL command to remove the Accept-Encoding header. We could actually build a Rack middleware that does exactly that! What would that look like? We take the same app, and all we do is modify the environment before sending it to Sinatra—the next link in the chain. As Matt said earlier, Ruby is awesome; it’s straightforward! You can just use `m.delete('Accept-Encoding')` to remove that header.
00:11:17.760
Another simple example could involve adding timing information. We might want to time how long our Ruby web app takes, and we can do that pretty easily. First, we can use multiple assignments: `status, headers, body` to take those three array items and put them into variables. This allows us to wrap the request in `before and after`, enabling us to calculate the duration and merge it straight into the headers.
00:11:34.360
As you can see, it reads really nicely. You’ve got to love Ruby! This highlights the possibility for change, either direction, and the possibilities here are essentially endless. For example, we could add or remove cookies; we could serve requests from a cache without bothering our Ruby server, or, and I’m guessing this is why most of you are here today, we could build CP Squ ourselves!
00:11:50.000
Alright, let's figure out how to do that. We’ve got our specifications from earlier—let’s build it in Rack! This requires adding one more implicit requirement: we’re going to use Rack middleware to do this. So, first off, let’s add some Rack middleware to an existing app and confirm it’s working.
00:12:09.440
I'm using Rails for this example, but it's fairly similar in other frameworks. The first thing I did was put some `puts` debugging in so I know it’s being loaded and called, then I placed that code in the `lib/middleware` folder—a case of convention over configuration. Once I did that, I modified `config/application.rb` to require that file and to insert the middleware at the head of the queue.
00:12:28.080
I used `config.middleware.insert_before 0, MyCustomMiddleware` to put it right at the top so that nothing else processes the request before I get to deal with it. Also, don't forget to relaunch the app anytime you make a config change or update your Rack middleware—ask me how I figured this out! After I finally got it working and booted it up, I saw the message I was looking for in the logs.
00:12:44.160
When I passed the request along, it logged all of the expected outputs. So now our Rack middleware is working! Now, how do we check for a coffee scheme? To accomplish this, we need to check the environment under `rack.url_scheme` to see if the scheme matches. If it does, we enter our block where we unescape it back to Unicode so we can read it.
00:13:02.560
We log it, then return a placeholder HTTP response to say, 'Got your coffee request.' If it doesn’t contain the coffee scheme, we just pass the request on to the next link in the chain. Once that’s done and we rebuild our server, I made some test coffee requests, and they all worked in different languages. To make sure I hadn’t broken my actual app, I sent a regular request as well, and Rails handled it as expected.
00:13:18.480
In the logs, I can see the three requests processed by my middleware and the one that got passed onto Rails. Alright, looking good! There are only four more requirements to cover. In the interest of time, we won’t go through all four; I’ll just show you one because they’re all actually quite similar.
00:13:34.640
For the first requirement, we want to check for a brew or POST method; otherwise, we will return a 405 method not allowed. We do that by adding a guard clause, which returns early if there’s a problem with the request and returns a 'Method Not Allowed' error unless the HTTP method is valid. It reads just like English, really nice!
00:13:50.960
How do we actually verify that? Again, we check if 'request.method' is set, looking for either Brew or Post. If it’s not set or is neither Brew nor Post, we return false, triggering the clause which returns the HTTP error. Returning this error is just like any other HTTP response; we need that three-item array. So for 405, we add our headers to indicate what is allowed, so the client knows what it should have sent.
00:14:07.440
We can put a human-readable string in the body in case anyone is reading it. That’s the first requirement. The next three are basically the same—three more guard clauses and three more different error types. At this point, I would have loved to show you a working coffee pot control server, but when I pitched this talk, I vastly overestimated my electronics abilities and time.
00:14:21.760
However, I am hoping to complete it one day; maybe there will be a sequel to this talk. For now, I will wrap up by covering what we discussed today. We looked at HTTP with its request-response cycle and some common methods, how URLs work, what headers do, and the different response buckets or categories.
00:14:32.720
We then looked at CP Squ and how it extends HTTP. While that may not be super relevant to implementing this, if you’re working with web dev or doing anything else that extends HTTP, it might come in handy. We also examined Rack—how it works and how it integrates different web servers with different frameworks.
00:14:49.680
Finally, we learned that you can have Rack middleware that lets you chain different requests together, and we implemented our own Rack middleware from scratch to handle the error checking portion of CP Squ. We learned how to integrate it into an existing Ruby web app. That’s all I’ve got for you today! Thank you for your time, and thank you to the organizers for putting together this conference and to all of you for being part of the Ruby community.
00:15:06.480
I hope you enjoyed yourself, and I wish you all the best for the rest of the conference.