Testing HTTP APIs in Ruby

Talks

Shai Rosenfeld

Testing HTTP APIs in Ruby

by Shai Rosenfeld

The presentation titled "Testing HTTP APIs in Ruby" by Shai Rosenfeld at Rails Conf 2013 discusses the challenges and methodologies associated with testing server/client HTTP APIs, particularly in a distributed systems environment like those at Engine Yard. The speaker emphasizes the significance of APIs in modern software development, akin to the importance of websites in the late 1990s.

Key Points Discussed:

The Importance of APIs: The importance of having an API for products in the current software landscape is compared to the essential nature of websites in the past.
Testing Methodologies: Rosenfeld outlines various approaches for testing APIs, such as:
- Isolated Testing: Testing server and client independently, which does not guarantee integration compatibility.
- Sandbox Approach: Enables running tests against a real server setup but may introduce complications like longer test run times and conflicts due to shared data.
- Fake Servers: Proposing a fake server that mimics the real API externally, allowing fast, low-dependency local development.
Cistern Framework: Introduced as a client API framework for hard coding mock responses, facilitating effective local testing while ensuring consistency with the actual API responses.
Shared Codebase for Mocks and Real API: A methodology to unify both the fake and real API functionalities and reduce redundancy in code management.

Illustrative Examples:

A fictional scenario involving a music labeling company where the speaker illustrates the roles of different developers and how they interact with the API through a client library.
Practical implications of each testing strategy were highlighted by detailing pros and cons, with specific emphasis on the real-world impact of each methodology in a development workflow.

Conclusions and Takeaways:

Testing APIs is inherently complex, and there is no one-size-fits-all solution; each approach has its own trade-offs.
Validated mocks can greatly enhance confidence in both client and server interactions, thus improving development efficiency.
The evolution of the testing methodologies shared aims to harmonize the needs of the client and server developers while ensuring robust API functionality and user experience.

00:00:12.259 Thank you.

00:00:15.859 So, uh, this is me. I work at Engine Yard. It's my personal blog website. I'm not sure why I put that there. That's me on Twitter; you can find me on the internet at shyguitar.

00:00:20.760 I'm going to start by reading this tweet that I liked a lot: "Not having an API in 2012 is like not having a website in 1998." The reason I like this tweet a lot is because I really think it summarizes where we are in the industry right now. You know, in the last bubble, we were all about having the web, websites, and putting your bar on the web or whatever. Now it's all about software as a service, infrastructure as a service, and platform as a service.

00:00:35.399 I even had a roommate who had razors as a service. The whole point of these services is really to have your product with an API that customers can actually interact with. Right? So, I'm going to be talking about how we test these APIs that we build.

00:00:58.559 Specifically, the problem statement I'm going to discuss is testing the server and client API. I’m going to be talking about testing the server and client that we build in conjunction together. This talk actually came out of work done in a very large distributed system where different systems need to communicate with each other, and they do so through APIs.

00:01:17.159 So, we will have each system talking to each other via an API that it exposes, and a specific client that will consume that API. The other system will use that client to communicate with the other system.

00:01:39.600 Now, I have this guitar with me and you may be wondering why I'm mentioning it. I want to set the context for this talk; I’m going to be talking about a music labeling company. So when I say 'we,' I mean this music labeling company. I'm going to introduce the characters quickly: we have a server developer who likes coffee, a client library developer who does a lot of work, and an app developer consuming that client library.

00:01:58.560 The app developer is not going to be part of the company; he's going to be a musician on the side who wants to consume our API. We’re going to expose a pretty simple API. Our domain model will have a title, which is going to consist of a unique name and associated words. We will post a song with a title and words and retrieve those details.

00:02:24.360 We aim to build a client for that API so that consumers can easily interact with it. Our goal is to create these endpoints on the server, build a client, and ensure that consumer applications can easily test with that client library.

00:02:45.099 Just for curiosity, I'd like to know: how many here have used Fog or know about it? Great! And how many have interacted with the mocking system that Fog provides? Not quite as many, but that's okay. Essentially, Fog allows you to interact with clients in a kind of test mode, which is essentially mocking.

00:03:09.180 You can call client.mock, and then you can interact with what's going on in the client, receiving server-like behavior even when no real requests are made. I will touch back on that later in the talk.

00:03:35.760 To reiterate our plans: we will create an API musicians can use to store songs with unique titles and names. We will build a client library for consumer applications and ensure it's easy for these applications to test using the client library.

00:03:59.040 Now, how do we actually test APIs? Small caveat: these are terms I've made up. There are isolated approaches, sandbox approaches, fake servers, and mappers. I'm going to explain these techniques that I’ve encountered while developing APIs.

00:04:18.820 I will start with the isolated approach. The isolated approach is the most natural progression: you code the server, you test the server, you code the client, and you test the client. Quite straightforward, right? But you often overlook this integrated testing chain between a third-party app and the server.

00:04:43.740 You might think, 'I'm just going to start coding,' and in the server, you'll end up with code that looks like this. The server will have a POST endpoint to create the song, store it in the database, and respond with some JSON output saying it started okay. This example is simplified, as there will obviously be more logic involved.

00:05:00.720 Similarly, the GET method will look up the song in the database, render back the words as JSON, and send it to the client, who will confirm receipt. Testing the client would involve asserting that when you call and post specific data, it returns the expected words.

00:05:21.300 However, the problem with this approach is that to test the client, you need to mock the server. You’ll use libraries like fake_web or webmock. Mocking out the server creates a potential issue; if the server changes, the client tests would still pass since they aren't actually interacting with the live server.

00:05:42.539 This becomes problematic if the server guy changes something in the API, and then the client guy ships their code. The musician who picks up the client discovers their app is broken while the client developer insists their tests have passed.

00:06:06.900 We realized we can't simply use webmock and wanted to run the actual code against a sandbox. We can host the server anywhere, even locally, and the idea is to run the client against the live server to ensure tests actually fail if the server changes.

00:06:21.300 The downside of running tests against a real server is that setting up a sandbox can be time-consuming. Platforms like Heroku and Engine Yard assist with sandbox setups, but it overrides your development process, especially if you’re frequently building new releases.

00:06:49.560 Another issue is making real requests could lead to unique data conflicts. For example, if your song's title is supposed to be unique, running the test a second time may fail because the server-side validation sees the title already exists.

00:07:05.520 There are advantages to running against a sandbox—setting it up allows a degree of confidence—that you’re testing the real API, but it's not without its hurdles.

00:07:28.680 The trade-off of this method is real-time requests incur substantial delays. An example is a partner project where tests took over an hour compared to a mocked setup that took only a minute and a half.

00:07:50.520 We're traversing the stack and testing various endpoints, but the length incurred from all this testing is considerable. You'll run into unique data conflicts too, creating a testing environment that's difficult to manage.

00:08:12.600 We need something that's quicker. This example led us to pursue a fake server approach.

00:08:35.100 The fake server method involves creating a server where we've outlined its expected responses, thus making the API simpler. Essentially, what's being done is that we simulate the server using, for instance, an in-memory hash that holds the expected behavior.

00:08:48.339 The fake server mimics the initial API setups, making it beneficial due to its simplicity while still validating the API calls effectively. You'll still ensure it acts as required, but you scale back complexity.

00:09:03.720 This procedure indicates how you can confirm the endpoint paths, return corresponding data, and validate the outputs without real server overhead.

00:09:22.740 All in all, our API’s logic can still be validated yet operates with speedy in-memory performance. The underlying logic remains, but you save time significantly.

00:09:40.380 Additionally, by running these tests locally and against a CI server, you create flexibility in your update processes. This offers confidence in deploying your code.

00:10:01.080 Specific libraries like 'Cistern' can help speed things up by mimicking server responses quickly. You validate the response you create, affirming it is indeed effective.

00:10:14.280 To summarize, I covered my experiences and lessons learned regarding server-client interactions in API development, touching on how to validate the API through fake and real interactions.

00:10:44.580 This consolidated perspective informs the approaches we can take and helps audiences understand the vast opportunities for improving their processes.

00:11:07.080 Ultimately, the best practices focus on validation, understanding API needs, and enabling clients to work seamlessly with them.

00:11:32.520 I hope this overview encourages everyone here to consider new ways to refine their approach to API testing, exploring additional tools and methods.

00:12:00.079 There’s no one-size-fits-all solution; this field, just like any software discipline, requires optimal adaptation.

00:12:21.360 So with that, thank you for attending. I would love to hear your thoughts, experiences, or questions about API development and testing.

RailsConf 2013