(Re)Building a large scale payments system on Rails

Database Transactions

Michel Weksler

1 talk

#ruby-on-rails

#payment-processing

#activerecord

#database-transactions

#service-objects

#error-handling

#code-quality

(Re)Building a large scale payments system on Rails

by Michel Weksler

In this presentation at RailsConf 2015, Michel Weksler addresses the complexities and considerations of building a large-scale payments system using Ruby on Rails (RoR). He acknowledges that while Rails has some limitations and trade-offs, it is, in fact, a viable tool for developing payment applications, given certain precautions. Key points in his talk emphasize the critical needs for audit trails, error handling, parameter validation, and structured testing within payments systems, which require high levels of reliability and accountability due to their direct impact on customer finances.

Key Points:
- Audit Trails: Payments systems must maintain thorough documentation of transactions, detailing changes comprehensively to meet legal and business standards. Weksler discusses the challenges of relying solely on ActiveRecord callbacks for auditing and explains the use of database triggers as part of the solution, albeit acknowledging their complexity.
- Error Prediction and Handling: He advocates for proactive error management to avoid letting issues escalate in production systems. Implementing clear parameter validations is essential to prevent unexpected situations from affecting operations.
- Code Quality and Maintainability: The importance of maintainable code is stressed, particularly for payment systems where quality is paramount. Weksler introduces the law of least knowledge to guide developers in structuring their code, reducing dependencies, and preventing unexpected behavior changes from affecting payment functionality.
- Testing Strategies: He encourages the use of smaller, focused classes and methods to enhance test coverage and clarify functionality. Service objects are highlighted as useful tools for encapsulating functionality and promoting clear input-output interactions.
- Straightforward Design Principles: Applying principles like DRY (Don't Repeat Yourself) aids in maintaining clarity and flexibility in the codebase. Weksler encourages well-defined methods that centralize access to data, facilitating easier maintenance and scalability.

In conclusion, while the challenges posed by developing a large-scale payments system are significant, leveraging structured practices around auditing, error handling, code quality, and testing can greatly enhance the robustness and reliability of such systems. Weksler's insights are drawn from his experience working in the payments sector, emphasizing the vital role of thoughtful software design in these sensitive applications.

00:00:11.840 My name is Michel, and I am an engineer on the payments team.

00:00:17.119 When reading back on the description for this talk, I thought, wow, people must be wondering what kind of talk this is going to be.

00:00:22.960 If you read the description, some of you might wonder if I'm going to complain about some aspects of Rails.

00:00:34.320 You know, the kind of complaints you often hear from people who haven't yet achieved nirvana as true Rails masters. I definitely have not achieved that, and I will be playing a little bit. But I hope that by the time I'm done, you will agree with me that some of those trade-offs are worth making.

00:00:52.239 I'm going to share my experience building, or more accurately, participating in the building of a few different payment systems for several large companies. Most recently, at Airbnb, I've tried to take some of those experiences and turn them into best practices to share with you. Even though I might complain a little bit, I started my journey not really thinking that Rails was the right tool.

00:01:17.720 I have to admit that I am now confident that this is the right thing and that using Rails for payments is doable, with some caveats that I'm going to discuss. I want to start over a decade ago when I joined what is now the largest successful payments company, which was then a startup acquired by one of Silicon Valley's successful companies. That startup was in the middle of hiring lots of engineers to help it sustain its growth and create the foundation for the future.

00:01:57.280 Pretty quickly after joining, it became clear to me that there was a ton of work to do, not just because there were lots of features to implement, but because the folks there did not have the time or the inclination to focus on long-term thinking. It's not that they lacked intelligence or motivation; they were simply focusing on survival and growth. What it felt like to me was coming home to find that your roommates had been partying all night, leaving the house in complete disarray.

00:02:30.400 They emptied the fridge, left a mess in the kitchen, and tons of production. If you Google around, you might find a few relevant examples from Airbnb's past, but that's a topic for another day. The kitchen now needs to produce a lot of gourmet meals for a very long time.

00:03:03.680 This brings me to my next point: any tech company, no matter which phase it is in, can and should think about code quality. You should think about stability, maintainability, and all of those factors, especially for companies or organizations within companies that handle payments. Customers have a special relationship with their hard-earned cash. They might tolerate outdated search results or social media sites being slightly off, but they notice and get upset very quickly if something goes wrong with their transactions.

00:03:45.280 While it's possible to iterate on various products and weed out problems gradually, payments is a little bit more difficult. This space is precisely where the bankers meet the hipsters, where payment requirements collide with underlying systems.

00:04:48.880 Let's start by discussing audit trails, which is one of the most basic requirements for any organization dealing with payments. It's essential to keep track of every change in business balances, documenting who made that change, why they made it, and the context of that change. Changes could originate from users interacting with the site, banks returning or completing transactions, or customer service agents assisting users. All of these actions are legitimate, but they all must be thoroughly documented.

00:06:20.319 While it seems simple to just attach some active record callbacks and declare 'you're done,' that's not entirely accurate. You need to ensure that you can guarantee to a judge or an auditor that 100% of those changes are being audited without exception, which makes the idea of using active record callbacks not so straightforward. How do you deal with the potential for any part of the code to call methods that could bypass your auditing?

00:07:18.240 This is possible in other languages as well, but Ruby is a little different due to its unique flexibility. While that's great in many cases, it creates challenges when trying to eliminate unwanted calls in your code. With active record, you get a lot of features automatically, but when it comes to transactions, it's possible for developers to bypass auditing methods.

00:08:10.840 At Airbnb, part of our solution was to use database triggers. This allows us to delegate the work to the database and lets it handle the heavy lifting, but it does introduce its own complications. Database triggers can be hard to manage and complex to migrate. Therefore, we're experimenting with application-level solutions where we create more stringent access on payment tables while allowing the rest of the application to use active record and all its benefits. One of the things we're trying involves creating a protected access gem that works similarly to active record but internally hides the model, making it harder to access the active record model directly.

00:09:57.760 While this limits convenience, it ensures that any access to the active record model must be explicit, thereby documenting the access and promoting discussion about whether external changes to the tables are necessary. This introduces a layer of discipline in handling our database interactions.

00:11:01.200 Next, I want to talk about error prediction and handling. The errors I'm referring to are not the ones caused when users interact with the system, such as typos in forms, but those that stem from the system itself failing. This might relate to unexpected states in the database.

00:11:21.760 Rather than stopping the application when an unexpected situation arises, I think it's essential to handle these failures gracefully rather than letting them linger, like driving down the road with a flat tire. Once you experience a flat tire, it's crucial to respond quickly to avoid damage to your vehicle.

00:11:54.240 In the application world, the standard approach involves immediately notifying someone or halting operations when encountering an error. While it's important to stop operations when necessary, we must also ensure that such situations are managed effectively.

00:13:09.200 Parameter validation is crucial here. We must ensure that our code includes checks for valid parameters and raise exceptions if unexpected states occur. This proactive approach helps identify issues before they cause more significant problems.

00:14:00.000 As we define our validation methods, it's essential to ensure that they remain clear and manageable. Keeping validations separate from the business logic can help avoid clutter and allow developers to focus on the core functionality.

00:15:33.600 In coding, we strive to protect our database from unintended changes, maintain an audit trail, and validate parameters. However, despite these efforts, issues may arise during production. One factor we consider when evaluating code quality is the law of least knowledge, which encourages developers to limit dependencies to objects that they are directly interacting with. This principle helps prevent unexpected changes in behavior if something upstream changes.

00:17:28.760 For example, consider accessing nested attributes within an object. If we directly reference a nested object, any changes to that structure might impact the existing code's functionality. Instead, we should encapsulate access through methods to minimize dependency.

00:19:09.440 Understanding when to change objects is critical. We should not make assumptions about whether the structure exists; instead, we should rely on defined methods to access information. This limits unnecessary tight coupling and provides clarity.

00:20:01.440 Next, I want to discuss testing. Instead of getting into specific testing frameworks, I'll focus on the idea that smaller classes and methods tend to lead to better tests. Testing large files or models is often challenging, which can lead to insufficient test coverage and resulting bugs.

00:21:56.320 Smaller, well-defined methods and classes not only make testing easier, but they also encourage clarity in understanding code functionality. At Airbnb, we utilize service objects to encapsulate functionality and ensure that we can focus on isolated behavior rather than extensive models.

00:23:00.580 Service objects help simplify interactions, ensuring each class performs a single task while promoting clear input and output definitions. They aid in keeping validation and functionality separated, enabling more focused testing.

00:23:50.680 Moreover, the ABC (Assignment, Branch, Condition) complexity metric can help assess the complexity of methods. By aiming to simplify code when the ABC complexity is high, we foster maintainability and improve easiness of understanding.

00:24:57.600 Additionally, we adhere to the Don't Repeat Yourself (DRY) principle, ensuring no knowledge is duplicated in code or data throughout our system. Each piece of knowledge should have a singular, unambiguous representation to avoid complications.

00:26:20.560 In practice, this means creating methods that encapsulate access to objects and data, allowing centralized updates without affecting other parts of the application. This enhances flexibility and avoid maintenance headaches.

00:27:39.680 In conclusion, while not everything mentioned here may be relevant to every application, integrating these strategies can significantly enhance payments systems due to their unique requirements. Using thoughtful metaphors that reflect practical experiences can enhance our software designs and improve our understanding.