Talks

Building Better Web APIs with Rails

At some point every Rails developer needs to build an API, but what's the best way to do it and avoid the mess many run into when serving multiple client applications?

In this session, we'll go over how to use Rails to leverage the HTTP protocol and build rock-solid web APIs. This includes fine-tuning route endpoints, content negotiation, versioning and authentication. We will also review common practices for keeping your code organized, clean and testable.

Ancient City Ruby 2015

00:00:00.080 Hello, everybody. I'm actually very disappointed. I just need to get that out.
00:00:06.000 When I knew I was coming to Ancient City in Saint Augustine, I thought, 'That's cool!'
00:00:12.400 Ghost town, high-end hotel. Maybe I'll run into a lifetime hero of mine.
00:00:19.039 But that hasn't happened yet. Jokes aside, I'm really happy to be here. I want to thank Hashrocket for putting this together.
00:00:25.039 It's always good to hang out with old friends and make new ones.
00:00:31.199 It's great to talk to people that are way smarter than I am. Like Mike. My name is Carlos, and I work for Code School.
00:00:36.239 We're located down the road in Orlando. We're an online school for learning web technologies: Ruby, Rails, JavaScript, and Git.
00:00:42.840 This week we launched our first SQL course—another free course. Try SQL! So, if you want to learn about SQL or have a friend you'd like to incentivize to learn SQL, make sure to check that out.
00:00:49.440 Also, we have a booth here where Christina, Dan, and I are hanging out. If you want to try out Code School and maybe get two days for free, make sure to stop by the booth.
00:00:55.199 You can sign up on our app; we promise not to spam you, but you'll get those two days for free. Now I want to ask you a question.
00:01:02.000 Does anybody here not work with Ruby full-time? A couple of people. All right, cool! So, stick around.
00:01:09.280 Later today, Dan is going to be talking about making the transition to working with Ruby in a startup environment during a lightning talk.
00:01:15.680 I know there are companies hiring, so if you'd like to work with Ruby full-time, maybe talk to those companies—Hashrocket might be one of them.
00:01:22.080 Now, I want to ask you to try and picture a specific situation that a few of you might have gone through.
00:01:27.280 This is the perfect Rails web app. It met the deadline and was live when promised, and it was developed within budget.
00:01:34.000 Your team didn't spend any more money than it had to—maybe even less than initially thought.
00:01:39.759 You have all the features that were agreed upon for launch day, and it has enough tests—not too many and not too few—so decent test coverage.
00:01:46.159 And it launched. It went live. You start getting new users into the system, and your client is happy.
00:01:53.600 Then a couple of weeks go by, and they come back to you and say something like, 'We need a snappier experience.'
00:01:59.759 You're like, 'A snappier experience?' What this typically means is that the client wants to push things a little more to the client side.
00:02:05.040 Maybe use some sort of client-side framework like Ember, Angular, or React to enhance the website and make the UI more responsive.
00:02:10.080 It's happened—you launched the website with certain expectations.
00:02:16.400 But it turns out that your users end up using it in a different way than you initially expected.
00:02:21.680 In this case, the users are accessing your website through mobile devices rather than through a desktop browser.
00:02:27.120 So your client realizes that they need to invest in a native app as soon as possible.
00:02:32.160 This is not bad; this is good! This is a good sign, especially if you follow the Agile and Lean principles. You embrace change—change is good, and you adapt to change.
00:02:37.680 So the natural next question is: Does our web app expose an API?
00:02:44.400 Can we create a native app that communicates with our original application without forcing us to rewrite or duplicate things in our web app?
00:02:51.040 The answer to this question is usually 'kind of,' because if you think about it, it's almost impossible to write a Rails web app without exposing some sort of API.
00:02:57.440 But is that API good enough to serve different clients? Maybe your JavaScript client, a native client, and even a command line application all at the same time?
00:03:02.640 What I'm going to talk about today is five things we can do to our app to confidently answer the question: Does our web app expose the web API? Yes, it does!
00:03:08.319 So we can write new API clients and seamlessly plug them into our existing app without a whole lot of work. Let's jump right into it.
00:03:14.959 First thing: routes. Routes are the entry point to our application, to our API.
00:03:20.720 The URLs are literally the API; they serve as the interface to our application, making them extremely important.
00:03:27.680 Good and expressive URLs are useful for both developers and API consumers, as they convey intent and serve as self-documentation.
00:03:36.560 For example, suppose we come across a URL for '/projects.' It's intuitive that this URL will return a collection of your projects.
00:03:44.479 If you make authenticated API calls to this URL, you would expect a collection of projects to be returned in the response.
00:03:50.720 We can also use URLs to filter results by adding keywords to existing root endpoints.
00:03:56.960 If '/projects' returns all of your projects, then perhaps '/projects/archived' will only return your archived projects.
00:04:04.000 Similarly, you can use an action like 'archive' with an identifier for a specific project to act upon that resource and archive that project.
00:04:10.080 These are straightforward examples of intuitive URLs that you don't need a lot of documentation to understand.
00:04:16.000 This makes it user-friendly and comprehensible for API consumers. So how do we implement something like this in Rails?
00:04:20.560 This is simple. If you've written any sort of Rails application, you've probably written something like this.
00:04:31.520 You use the resources method, which has been around since Rails 2.0, back in 2006.
00:04:40.480 So, you would write 'resources :projects.' This one line in your routes file will give you a variety of routes.
00:04:46.880 These routes, combined with a subset of the HTTP methods, will cover all combinations for controller actions—the CRUD operations: create, retrieve, update, and delete.
00:04:52.399 You get all of this out of the box with just that single line. But what about those two filtered URLs we just looked at?
00:04:58.640 What if we wanted '/archived' for the archived projects? Well, we can use two methods—collection and member.
00:05:02.240 As the name implies, 'collection' refers to new endpoints that act on the entire projects collection, while 'member' refers to a single project.
00:05:10.080 The issue is that once people learn they can do this, they start adding it for everything.
00:05:18.179 This bloats the routes file, but the real problem is that we'll end up with fat controllers.
00:05:25.919 Each additional route translates into a method in that specific controller, so the projects controller will have not just the seven default actions.
00:05:31.120 It will also include archived, active, suspended, etc., turning the controller into a huge mess.
00:05:38.080 Fixing this is relatively easy. You just need to add another option to each method and specify the controller action that should handle that URL.
00:05:44.560 So, the URL remains the same, allowing us to keep expressive URLs while effectively directing them to specific controllers.
00:05:51.040 For example, the 'archived' action URL will now be directed to the archived projects controller's index action.
00:05:56.720 Likewise, 'active' will refer to the active projects controller and 'suspended' will map to the suspended projects controller.
00:06:03.040 You may think that’s a lot more controllers than you had before. Yes, it is, but it allows you to maintain the default actions for the CRUD operations.
00:06:10.080 If you consider this approach, it's actually easier to think about if you just want to get a list of archived projects.
00:06:18.080 You’re literally listing archived projects. It makes sense, and it helps you spread out the logic across different controllers.
00:06:25.680 When you have developers working on something related to archived projects, it's unlikely they will conflict with another developer's work.
00:06:31.200 Now, how about the last route, 'create review'? This one is a good candidate to become its own resource, hence 'resources :reviews.'
00:06:39.760 However, if we only needed to create reviews, we could narrow it down to create.
00:06:46.560 Reducing resource definitions like this helps with performance, so your app runs faster.
00:06:52.800 Next, let's talk about content negotiation. Content negotiation is how the client and server agree on the best format for a response.
00:06:59.680 You can have the same backend code that responds to different clients—perhaps JSON for one specific application, XML for another, and HTML in a different language for yet another application.
00:07:05.840 All of this can come from the same code base.
00:07:11.296 So how do we serialize our API clients back effectively? The built-in way in Rails is to use Jbuilder.
00:07:18.240 It ships with Rails; it's already in your Gemfile—though it might be commented out. This is what the syntax looks like.
00:07:24.320 It's beneficial because it's maintained under the Rails organization. One of the maintainers works for Hashrocket and may be around here.
00:07:30.080 I’ve used it several times. However, some people feel uncomfortable with the syntax. Another popular option is ActiveModel Serializers.
00:07:45.849 This project is a little more object-oriented, allowing you to write serializers in a more OOP approach.
00:07:51.760 The syntax here mirrors ActiveModel, making it easy to use relationships at the serialization level.
00:07:57.760 You can declare your attributes and do some cool stuff, like embedding IDs, which help speed up your client side.
00:08:05.840 This is extremely useful, especially if you're using something like Ember.js with the JSON API adapter. It helps do a lot of work for you and saves on additional HTTP requests.
00:08:11.760 ActiveModel Serializers was originally created by Steven Klavnick, who is an amazing individual involved in many different projects and has many commits to Rails.
00:08:19.760 Currently, this project is undergoing a major rewrite. If you want to use ActiveModel Serializers, take a moment to read through the documentation to see which version works best for you.
00:08:26.560 It's still a good choice; I've used it quite a lot. The coolest aspect of ActiveModel Serializers is that it defaults to the JSON API format.
00:08:35.120 The JSON API is a media type also created by Steven and a few others. It's a standard for building APIs in JSON.
00:08:43.680 What this means is that it's an anti-bike shedding weapon, stopping discussions about trivial matters and allowing you to focus on what's important.
00:08:50.560 JSON API suggests formats for returning collections, single resources, and declaring relationships between your resources.
00:08:56.719 This is important so that developers don’t end up arguing about how to structure their resources.
00:09:06.720 If you follow the JSON API format, you can skip the discussions and focus on business logic.
00:09:11.120 You might find that if you use Ember, the REST adapter also follows the JSON API schema.
00:09:15.680 Setting your Rails application to serialize as JSON API will make things seamless for you.
00:09:22.560 Another option is to use a project called Roar, which is the newest of these solutions, developed by Apotonic.
00:09:30.560 This project is a wrapper around another project called Representable.
00:09:36.960 The syntax looks pretty simple and gives you two options: you can either declare your serialization form as a module and mix that into your pure Ruby classes in your Rails app.
00:09:44.160 Or you can take a more object-oriented approach and simply write a class that extends Roar's decorator.
00:09:52.400 Another cool feature is that you can also use it to parse incoming messages, allowing you to write both the server and the client using Roar.
00:09:58.760 Now I want to ask you to try and read this message.
00:10:06.080 Alright, maybe this one is easier.
00:10:11.760 Not missing a semicolon at the end. How about this one?
00:10:16.000 A little bit better, right? These three mean the same thing, but it's hard to agree on their meanings.
00:10:23.680 When we talk about APIs, we want something that's easier to decipher than messages like these.
00:10:30.080 For instance, consider HTTP status codes. If you've worked with APIs long enough, you know that a 201 status means the request was fulfilled and a new resource was created.
00:10:37.760 A status code can provide more clarity than a long message, so it's extremely important.
00:10:44.480 Before you even think about what type of error or success message to return to your client, focus on the status code.
00:10:51.280 An API client that is thoughtfully designed will first look at the status code of the response to determine what to do next.
00:10:57.440 Whether the response was successful, a failure, or a redirect, it all depends on the status code.
00:11:03.840 Now, let's look at a recent feature added in Rails 4.2. Suppose we have an endpoint that generates reports.
00:11:11.360 This takes a POST request that generates a report and returns a 200 response with an empty body.
00:11:19.760 This is what the HEAD method means: it simply returns a 200 status code with an empty response body.
00:11:25.120 However, report generation can take a long time, say 20 seconds.
00:11:32.960 You don’t want to leave that request waiting for 20 seconds while the report is being generated.
00:11:39.360 Instead, you want to move this processing to a background job.
00:11:44.960 In Rails 4.2, you can do this using Active Job. You simply put the time-consuming code in the perform method of your job class.
00:11:52.480 You would call this job in your controller, replacing 'report.generate' with 'ReportJob.perform_later.'
00:11:58.960 This will add the job to the queue, returning a response to the client as soon as the job is queued.
00:12:06.160 However, this is not correct because it's returning a 200 status.
00:12:11.600 A 200 status means that the request was completed—that everything the client wanted was fulfilled—which is not true.
00:12:18.560 Instead, there’s a better status code for this situation: a 202 status, which means it has been accepted for processing, but processing has not yet been completed.
00:12:25.280 It's crucial to communicate to your API clients what is happening on the server.
00:12:32.960 Now, let's discuss authentication. There are different ways that Rails supports authentication out of the box.
00:12:40.480 But I want to focus on one method that is most commonly used for APIs: token-based authentication.
00:12:47.680 One advantage of using tokens is that you can easily expire them.
00:12:54.080 If you’re reusing credentials for a website and you expire those, the user can no longer access the site. However, if you provide a specific token, you can deactivate it without affecting the user's overall access.
00:13:02.920 Moreover, different tokens can be generated for different services, allowing users to shut down specific tokens in the future if needed.
00:13:09.360 You can also implement different access rules for each token, giving more control when using token-based authentication.
00:13:15.240 How do we provide the token as API developers? It typically involves an out-of-band process, meaning it's not part of an API call.
00:13:23.440 Usually, users will have to navigate to the website, often to their profile page, to retrieve the token, and they should not keep it in version control as mentioned earlier.
00:13:29.840 From the Ruby side, Rails offers an easy method for authenticating a user through the token.
00:13:34.960 You can use the 'authenticate_or_request_with_http_token' method, which takes the token as an argument to your custom authentication method.
00:13:41.440 If the method returns something truthy, then the request will continue; otherwise, it will respond with a 401 status.
00:13:48.480 Now from the API perspective, how do you generate this token and display it on the user’s page?
00:13:55.040 To generate the token, we can use UUID, which stands for universally unique identifier—a standard with an RFC to ensure broad support.
00:14:01.440 To create a UUID in Ruby, we call 'SecureRandom.uuid,' which will return a unique token.
00:14:08.080 To ensure uniqueness, you might want to create a database constraint, but we’ll skip that for now.
00:14:15.520 Once generated, you can move that logic into the user model using an active record callback.
00:14:22.080 In this specific instance, I use the ‘before_create’ callback to set the authentication token.
00:14:29.440 The method will return immediately if the token is already present, allowing for custom token testing.
00:14:37.280 Next, if the token is not present, we generate the token with our previously defined method, ensuring it’s URI-friendly by omitting any hyphens.
00:14:45.760 This is a simple approach in Rails, ensuring uniqueness and simplicity.
00:14:50.800 Testing your API can be accomplished using curl, which allows you to utilize the '-H' option to pass custom request headers.
00:14:57.760 As the token authentication conforms to the HTTP spec, you provide it using 'Authorization: Token YOUR_TOKEN.'
00:15:05.760 If you receive a 200 response, then you're good. If you use the wrong token or omit it, you'll see a 401 not authorized response.
00:15:12.960 Curl is a valuable tool when working with APIs.
00:15:20.480 Next, let's talk about versioning.
00:15:26.080 The first method that comes to mind when we talk about versioning is semantic versioning.
00:15:31.760 Semantic versioning is great for software and libraries, but not as effective for services.
00:15:38.960 In essence, semantic versioning involves updating the patch when you fix bugs, updating the minor version when you add backward-compatible changes, and updating the major version for backward-breaking changes.
00:15:44.560 For services, my advice is to focus solely on the major version.
00:15:53.440 You should try to avoid maintaining multiple versions of a service as much as possible because supporting various library versions is different from maintaining service operations.
00:15:58.480 When you have multiple versions of a service, you're responsible for keeping the infrastructure up and running.
00:16:03.360 Using only major versions means you only have to worry about breaking changes.
00:16:13.040 There's no need to bump the version on compatible changes; you can simply make announcements through blog posts or emails about the changes.
00:16:18.880 For example, if you add a new format, like XML, but your clients don’t care about it, or if you add a new property to a resource.
00:16:27.840 You can also rename endpoints using status codes within the 300 series, which deal with redirects. No need to force users to change their programs unnecessarily.
00:16:36.120 However, if you remove an endpoint, yes, that counts as a backward-incompatible change.
00:16:42.560 Even if you return a status code indicating that a resource was removed, it's better practice to notify users that there's been a change to their API access.
00:16:51.040 I don’t think I would change the version in that case, but it's an option to consider.
00:16:56.960 Finally, let's talk about tests—how to automate the verification of your APIs.
00:17:03.760 The most important thing about testing APIs is that it's not the time or place to test business logic. That should happen in unit tests.
00:17:10.240 API tests should focus on a couple of key areas.
00:17:16.560 Testing statuses, MIME types, and authentication are crucial for your API.
00:17:21.040 The easiest way to test in Rails is through integration tests.
00:17:27.040 This approach is simple and does not require additional frameworks. Let’s consider a test for listing our projects.
00:17:33.568 If our API is set to run on a namespace or a subdomain, we call the host method with the subdomain, which in this case is 'api.'
00:17:41.440 We then issue a GET request to the endpoint '/projects,' asserting two essential conditions: the status code should be 200, indicating success, and the response body should not be empty.
00:17:48.400 This will take you far in terms of testing the core functionality of your API.
00:17:55.040 After that, you can move to more granular testing based on controllers or models, focusing more on unit tests if necessary.
00:18:01.280 Next, we want to ensure that our endpoints return both XML and JSON MIME types.
00:18:08.800 To achieve this, we utilize the accept header, requesting the type to be JSON first. You can use Rails' constant for application/json.
00:18:16.000 Then, you can assert that the response matches the expected content-type header.
00:18:22.560 The same applies to XML; the accept header must match the response's content-type header.
00:18:30.000 This straightforward approach makes it easy to verify desired API behaviors.
00:18:36.560 Now, regarding testing authentication and access rules, we’ll follow the same setup, making sure to include our subdomain.
00:18:41.760 For the user token, I usually let it be generated, but injecting it is also an option.
00:18:48.560 You set the authorization request header to that user's token, which is an authenticated call, and it should return a 200 status.
00:18:55.440 Additionally, you want to ensure that the response returns a JSON MIME type.
00:19:01.920 At the end, test with a fake token to verify that you receive a 401 unauthorized response.
00:19:07.760 This proves that even with a straightforward structure, you can ensure that your API functions as intended with permissions and access controls.
00:19:13.120 That concludes the core points I wanted to cover today.
00:19:21.440 I want to share a shameless plug for one of our courses at Code School—'Surviving APIs with Rails.' This course talks about what we've mentioned today and goes into more detail.
00:19:27.760 You can try it in the browser without downloading anything to your computer—check out the course if you want to learn more about API development in Rails.
00:19:35.840 Additionally, at Code School, we love podcasting. We do a bi-weekly Ruby 5 podcast and recently launched a podcast focused on JavaScript, called Five Minutes of JavaScript.
00:19:43.040 We appreciate having guests on the show. This is not an interview format; it's a host and co-host set of quick episodes. Each episode takes about five minutes to record.
00:19:49.760 The writing and recording process takes about an hour, and we don’t have to edit.
00:19:57.680 If any of you are interested in participating in these podcasts, please send me an email. I'd love to discuss it.
00:20:06.080 If you have any questions, feel free to pull me aside. I would love to chat with you.
00:20:12.480 Thank you very much for your attention today!
00:20:15.840 Thank you!