Talks
Testing Integrations: The Good, the Bad, and the Ugly

Summarized using AI

Testing Integrations: The Good, the Bad, and the Ugly

Julia López • September 26, 2024 • Toronto, Canada

The video titled "Testing Integrations: The Good, the Bad, and the Ugly" features Julia López discussing practical strategies for testing third-party API integrations. Based on her experiences at Harvest, where her team implemented several integrations with various services such as Stripe and QuickBooks, Julia delves into the challenges and solutions in this area. She underscores that the talk is not centered on the general importance of testing, but focuses specifically on the complexities of testing external API integrations. Below are the key points outlined in her presentation:

  • Definition of Integration: Julia begins by explaining what an integration is, emphasizing the role of APIs in connecting different software.

  • Importance of Testing: While acknowledging the significance of testing, she highlights that testing external APIs presents unique challenges that make it distinct from general software testing.

  • Common Problems: Julia lists several issues encountered when testing API integrations, such as:

    • Slow API calls
    • Rate limits imposed by APIs
    • Fluctuating data that impacts test reliability
    • Authentication complexities
  • Testing Approaches: She discusses various strategies for testing external APIs, including:

    • Live API Calls: Running tests that hit the actual APIs, which can be slow and problematic due to rate limiting and fluctuating data.
    • Stubbing and Mocking: Using tools like Mocha and WebMock to fake API interactions. This helps overcome issues related to live calls, such as network latency and variable responses.
    • Using VCR: Julia introduces VCR, a gem that records API interactions to replay during test runs, allowing for realistic testing without hitting real APIs on each test.
  • Real-Life Examples from Harvest: Julia shares insights from Harvest's integration testing strategy, detailing how they use VCR and mock responses to maintain efficient testing workflows without compromising reliability.

  • Conclusion and Best Practices: Julia concludes by summarizing the pros and cons of the different testing approaches. She emphasizes the importance of having a good test suite and ensuring that they continue to evolve with the integrations.

Overall, the key takeaway is that while integrating third-party APIs can enhance application features, testing these integrations effectively is critical due to the complicated nature of API interactions. Julia advocates for using a combination of tools and practices to ensure robust and reliable testing workflows.

Testing Integrations: The Good, the Bad, and the Ugly
Julia López • September 26, 2024 • Toronto, Canada

Enhancing your app's features through third-party APIs can be so powerful, but testing these integrations can be so cumbersome. At #RailsWorld Julia López sharaed practical strategies for testing integrations, drawing from real-life experiences at Harvest, where they implemented several integrations – Braintree, Stripe, CustomerIO, Hubspot, Xero, QuickBooks, and more.

#tesing #Rails #api

Thank you Shopify for sponsoring the editing and post-production of these videos. Check out insights from the Engineering team at: https://shopify.engineering/

Stay tuned: all 2024 Rails World videos will be subtitled in Japanese and Brazilian Portuguese soon thanks to our sponsor Happy Scribe, a transcription service built on Rails. https://www.happyscribe.com/

Rails World 2024

00:00:14.480 Today, I'm going to be discussing testing integrations: the good, the bad, and the ugly. I chose the title before I had any content prepared because it kind of rolls off the tongue, and people recognize the movie; I have not watched it myself. I hope everything fits together.
00:00:27.599 If you saw my talk at Rails World last year and enjoyed the animations, I hope this one does not disappoint! My name is Julia López, and I'm from Barcelona. I've been working with Rails since 2011. I love refactoring, upgrades, and everything that Robi mentioned before. I'm always pushing for that to happen.
00:00:50.760 I've added something new to my repertoire called billing, because for the last three years, I've been leading the implementation of a new billing system at Harvest, specifically integrating with Stripe. Just a heads-up: most of the examples I'll cover today will be about that because I wanted to focus more on the present rather than the past.
00:01:12.200 I work at Harvest, the time tracking tool, which has been built on Rails since 2006. We have a small engineering team, so I have had the chance to work on many parts of the code. It's a monolith, and we own everything; it's our baby, and even with an 18-year-old app, that's something to keep in mind when I go through examples today.
00:01:51.840 I want to thank the people who set up everything I’m going to show today. I have just been using it and extending it, but it wasn't my original idea. Harvest connects to multiple tools to extend the features we offer, like QuickBooks, Stripe, Microsoft, Google APIs, PayPal, Slack, and others for internal use like Stripe, Braintree, and CustomerIO.
00:02:23.360 I wanted you to understand that working with APIs is not always joyful; we've seen it all. So, what can you expect from my talk? This is not a discussion about why testing is important—it is important, as Robi mentioned—but rather a practical insight into popular Ruby gems for testing. My recommendations will be opinionated, and I'll show you the pros and cons through real-life examples.
00:02:59.839 I hope you leave knowing something new. You might already be aware of some topics, but I believe you will relate to our experiences and challenges. Let's start by refreshing the concepts for those who may not know: what is an integration? A software integration is connecting software with another piece via Application Programming Interfaces (APIs). APIs are the rules and protocols that allow software to communicate with each other.
00:03:51.840 There are many APIs, with the most familiar being HTTP and HTTPS. All examples today will deal with APIs that operate on these standards. We will specifically focus on third-party APIs. You could have your own front-end client with your own API, but that's not what we're discussing here. Testing, a broad term, refers to checking whether your software works as intended. We will specifically look into automated testing.
00:04:49.639 Automated testing means having software that tests your software—it's checking that your software performs as predicted. Why focus on testing external API integrations instead of general testing? Because you can conduct general testing with the tools I will discuss, but we cannot control third-party APIs. There are many issues to consider, and developers rarely express joy when working with these APIs.
00:05:40.319 Writing tests is challenging, especially for external integrations. Let's start with a simple billing integration example using the Stripe API for subscription creation. Imagine a service that will make a POST request to the subscription endpoint of the Stripe API. The subscription endpoint returns a response with various attributes about the created subscription.
00:06:27.720 In a simple code example, you have a create method that generates a subscription. However, where is the API call located? To be realistic, we're using the Stripe gem to implement our billing integration, which I highly recommend, but as Robi said, be cautious about adding too many gems to your stack.
00:07:05.560 If your use case involves only one API call, it's better to avoid unnecessary gems. Instead, utilize an HTTP client like Faraday whenever possible. For the sake of this presentation, let's pretend everything is functioning perfectly. Testing is essential when building an integration, so how do we approach this testing? There are different approaches, but remember that not testing is not an option. If you want to take risks, then that's a separate approach.
00:08:03.159 One approach involves making a live API call for every test run; that’s acceptable, but it can lead to complications. Let's look at some code demonstrating this concept. Here's a simple test definition using MiniTest to call the service and assert expectations. However, your test suite will likely be more complex than this example, especially when dealing with something as nuanced as a billing system.
00:09:25.480 You can easily accumulate hundreds of tests to run, quickly escalating in size. Consequently, you may encounter problems, such as slow API calls. For instance, I benchmarked the time it takes to invoke our service, and it took 400 milliseconds just to hit the API. No one enjoys a slow continuous integration process, especially when things are urgent.
00:10:40.319 Another problem is rate limiting, which many third-party APIs impose to prevent service overload. Stripe, for instance, has strict rate limits, especially in test mode. Finally, you also need to account for potential issues like poor network conditions, timeouts, latency, or interruptions due to maintenance or human error. Even the best services occasionally experience problems.
00:11:40.680 Another concern is fluctuating data. If you're using a production API for testing, often data will change, creating instability within your tests. What happens if a customer or pricing plan disappears? Your tests may break, hindering your development process.
00:12:12.080 Then there are issues related to authentication and authorization, especially if you are using OAuth flows which require token management. More API requests can lead to slower tests, and if you exceed limits, errors can arise. Then we have to ask, when is it safe to deploy?
00:12:33.120 As you can see, many things can go wrong. The solution? Avoid hitting APIs directly and instead use stubs and mocks. A mock is an interceptor in your tests that prevents API calls from executing. For example, if you're using `Mocha`, a popular Ruby library for mocking and stubbing, you would set expectations that your service calls the API using predetermined requests and responses.
00:13:23.680 Let’s revisit the previous Stripe example. If we employed the Stripe gem directly, we would mock the Stripe subscription method to set expectations and simulate responses. If we use `WebMock`, it allows setting expectations on HTTP requests without being limited to a specific client, which is beneficial for flexibility.
00:14:51.360 Utilizing `WebMock`, you can stop and set expectations on HTTP requests by specifying parameters. You can configure your tests to assert the body and response of the requests made, ensuring your integration behaves as expected. With both `WebMock` and `Mocha`, we ensure that our tests are not tied to live API calls, leading to faster and more stable test runs.
00:15:56.960 But what happens if I only need certain attributes? For instance, if Stripe returns 40 attributes, but I’m only mocking a few, how can I accommodate various scenarios like different subscription types? You end up needing a lot of boilerplate code to represent different states, especially since API responses can change.
00:16:50.480 What if I tell you that there's a better solution? Enter `VCR`, a gem that records your HTTP interactions during tests and replays them in future test runs. This makes it easy to manage our API calls by preventing unnecessary duplicates while also keeping our tests consistent.
00:17:53.240 When you utilize VCR, you need to tell your tests to use a cassette—a recorded file of your real HTTP calls. This way, the first time you run your test, it will hit the API, and subsequent runs will replay that recorded response, effectively eliminating repetitive API calls in many cases.
00:18:56.480 VCR's recordings preserve a substantial amount of information, including status codes and request details, but it is also crucial to ensure that customer IDs and price IDs used are valid and relevant, as VCR will re-use your cassettes at runtime. This allows us to work efficiently without additional overhead from API calls.
00:19:57.680 While VCR and `WebMock` work well together, the challenge is that we are managing a 17-year-old codebase here at Harvest, which makes some integrations difficult to modernize quickly. Nevertheless, VCR and `WebMock` can provide insight and ease in testing our API interactions.
00:20:25.240 Our VCR configuration is essential; it sets the folder where cassettes are saved and integrates with other libraries like `WebMock` to ensure that our tests remain inside expected behaviors without unexpected calls to outside services, especially during continuous integration.
00:21:11.960 In terms of maintaining a clean testing environment, we need to keep an eye on our VCR cassettes. For instance, when a major change happens, we need to recreate these cassettes, which can lead to significant file changes. This is an annoyance during pull requests, as they often come with a multitude of cassette changes.
00:21:55.880 Even though coding more tools might sound overwhelming, it actually streamlines our tests because VCR helps us avoid unnecessary API interactions. With proper documentation, new developers can acclimate quickly to necessary setups for authentication and token management. Challenges may arise, but so long as we pay attention, the setup remains effective.
00:22:46.720 Incorporating tools like `WebMock` and `VCR` helps us test integrations realistically while maintaining clarity in request and response visibility. We also have the flexibility to adjust our systems based on output without managing the entire system call logic. This is vital for maintaining and extending functionalities effectively, leading to better test suites.
00:23:47.480 While we have shared the challenges of working with APIs, from fluctuating data to complex setups, thorough testing ensures a smooth deployment pipeline. Keeping in mind the importance of effective testing will have a serious positive impact on our products and processes, encouraging faster iterations and more robust software.
00:24:34.760 To wrap things up, thorough involvement with both VCR and WebMock benefits us in multiple areas—from audit trails of API interactions to significant boosts in our testing speed. Remember that every integration will have its own unique complexities, but utilizing tools strategically will power through hurdles without sacrificing performance or reliability.
00:30:05.760 Thank you!
Explore all talks recorded at Rails World 2024
+17