Ruby Video | Designing a Great Ruby API - How We're Simplifying Rails

RailsConf 2015

Designing a Great Ruby API - How We're Simplifying Rails 5

by Sean Griffin

In the talk titled "Designing a Great Ruby API - How We're Simplifying Rails 5," Sean Griffin shares insights on the process of designing a simplified API for attributes and type casting within Rails 5.0, emphasizing that the most effective APIs are simple and composable. He discusses the importance of identifying missing APIs in legacy codebases like Rails, and shares strategies to refactor and extract simpler APIs from complex existing code. Griffin illustrates the complexity of modifying attribute types in Active Record, using the example of a price attribute that needs to be represented as a money object.

Key points include:
- Beginning the Refactor: A rough outline of the desired API is essential, avoiding early rigidity to allow flexible iterations.
- Identifying Duplication: Developers should look for duplicated concepts and patterns in the code, such as methods with similar prefixes or overridden methods in modules.
- Overriding Methods: The need for overriding readers and writers to create desired behaviors, such as converting attribute types or managing typecasting behaviors, can lead to challenges and subtle bugs.
- Introduced Complexity: Examples demonstrate that while trying to implement new features, existing hacks and complications can create unexpected behaviors.
- Refactoring Rules: Stressing the importance of good test coverage throughout the refactoring process ensures stability and provides a safety net for changes.
- Iterative Process: The talk stresses an iterative approach to refining APIs and type casting behaviors, where each change is validated with thorough testing.
- Clarity and Maintainability: As the new API develops, maintaining clarity in the implementation remains paramount to ensure usability and predictability.

Griffin concludes by emphasizing the need to balance complexity and functionality in API design, ensuring that enhancements in Rails 5 provide a streamlined and coherent experience for developers. He highlights that the ongoing challenge will be to evolve the APIs while keeping them intuitive and accessible. This talk aims to encourage developers to push the boundaries of Rails API design while maintaining a focus on developer experience and code maintainability.

00:00:12.320 Once we've identified what we want to build, we need to roughly— and here the keyword is roughly—decide what we want it to look like. One of the worst things you can do is pin yourself down to a very exact API too early. When you're refactoring towards a new API in an existing system, it's very important that you have good tests. The final steps of actually implementing it involve building those objects, using them internally, and composing them together manually where needed. The DSLs will come from where we find duplication or pain. Before you can design a great API, the first step is to identify the API that you're missing in any large legacy codebase. Make no mistake, Rails is just a large legacy codebase. All the strategies that you can use everywhere else in your code still apply here.

00:01:10.240 You can find plenty of concepts that are duplicated across the domain. Some of the smells to look for are methods with the same prefix, code that has similar structure, or the big one that you find in Rails a lot—multiple modules or classes that are overriding the same method over and over again and calling super. One of the concepts we found inside Active Record was the need to modify the type of an attribute. For example, say you have a price column on a product table, and you would like to represent that as a money object instead of a float.

00:01:37.120 To implement this, we can override the reader and writer, checking to see if anything is nil. We're effectively creating a money object in the reader and extracting the amount from the money object in the writer. When writing code like this, even experienced developers often wonder if they are going to break Rails magic. It actually kind of depends—you probably won't break anything, but there are some things that might behave unexpectedly. For example, we have a before typecast version of our attributes, and when you override the writer, you're actually doing some casting and then giving it to Active Record. It thinks it's before performance casting; the form builder expects certain values to be in a certain structure, and some validations expect things to be a certain way.

00:02:21.120 This might work a little bit differently than you want. Dirty checking also relies on before and after typecast values. Even if you don't break things, there might be other applications of this behavior that are just flat-out impossible today. For instance, you might want control over the SQL representation of your money object. Maybe you want to add in currency and store it as a string instead of a float, needing to parse that and combine them back together when sending to the database.

00:03:03.480 The really hard part right now is that you might also want to be able to use this object as a value in queries. Passing it to `where` is incredibly useful. Rails overrides types internally all over the place. You might be wondering, if this is so hard, how does Rails do it? If you guessed it's with a giant pile of hacks, then you would be correct. We're going to look at some Rails internals, not to dissect it entirely, but to illustrate how complex it can be.

00:04:01.959 The specifics of the code are not too important, but one feature that's on by default converts time values that you pass into Active Record to the current time zone. This is something that most people don't realize, but it’s enabled by default in most applications. We override the writer here while modifying several behaviors: first, converting to the current time zone; second, implementing dirty checking. The second and third lines in this method are copied from inside the dirty module in Active Record. After that, we jump through a lot of hoops to maintain the before typecast version of the attribute.

00:05:02.560 We're looking for common concepts and smells in our code. We are overriding the reader and writer, duplicating a lot of code. This relatively small behavior change requires complicated adjustments to be done correctly. It’s also important to note that writing code this way can introduce a lot of subtle bugs. Many other modules may be trying to modify the type of this attribute in unexpected ways, and bugs are hard to detect when behavior is spread throughout the code.

00:05:57.120 Another place we modify the behavior of the typecasting system is in serialization. We’re overriding a method that gets called internally to perform typecasting. Instead of explicitly overriding a reader and writer, this module complicates things. When we started, this module literally overrode every single method in Active Record that contains the word 'attribute'. There are five or more slides filled with code that I left out for brevity, but you get the picture.

00:06:55.199 The result is that we were duplicating code tightly coupled with Active Record. This was the cause of numerous bugs in versions 4.2 and earlier. When we look at enums, which represent integers as a known set of strings, we find similar issues where we overwrite both the reader method and the writer method. This process also introduces unexpected vulnerabilities. We found that typed attributes were overridden all over and if we need to implement this, other developers might want to do the same.

00:08:02.799 Type casting is when you explicitly convert a value from one type to another. A simple example is having a value, which is a string, that we want to convert to an integer. So we call `to_i` in Active Record. However, what we really do isn’t type casting; it’s type coercion, which is the same when done implicitly. For example, when using Active Record, we can have a user model where age is presumably an integer column in the database. Whenever we assign a value to the age attribute, we convert it to an integer. This design simplifies code across controllers but does have complications with types like dates.

00:09:14.040 We didn’t want this code to be littered across controllers, so the Active Record type system was established. The cases we handle today are much more intricate than before, but if you track back the history of its evolution, everything can be traced back to that initial limitation. Now in Rails 4 and earlier, the only way to have a coerced attribute is if it’s backed by a database column. We wanted to hook into this behavior for our refactoring.

00:10:14.160 In step two, we roughly identify how we want our API to look. We have a product model and know a few things: we need to call a method—in this case, we’ll use 'attribute'—and we need to specify the name of the attribute while marking what the desired type should be. This behavior aligns with APIs found in data mapping libraries or MongoDB but we will avoid over-specification at this stage. The nebulous part will be how we pass the type.

00:11:02.240 What we know is that we will need to introduce a type object into our system, but just that won’t yield a reasonable implementation. We’re looking for something we can be proud of and feel confident maintaining in the future. We will start by manually composing the objects in our system. The only known object is our type object, and we will be searching for opportunities to extract collaborators to ease our workload.

00:11:41.400 Before we begin introducing the API, a few brief rules of refactoring need to be followed. Rule number one: have good test coverage. Rule number two: also have good test coverage. Rule number three: see rules one and two. On the following slides, we’ll see that while the specifics of the code can be small, what’s important is acknowledging the overall structure. There’s a giant case statement that you would find in a part of Active Record handling type systems in version 4.1.

00:12:42.640 This statement would call class methods based on symbols derived from SQL types. As you glance at the top, there’s a misleading comment: 'tests value, which is a string to the appropriate instance.' This leads to confusion regarding how to pass different values that aren't strings, which is particularly misleading.

00:13:55.200 We know that we will introduce a type object, and type casting will now reside on the column. Our first step is to give the column a type object, so we add a constructor argument and pass it everywhere. Subsequently, we run the tests, solidifying this as our very first commit towards improvement, albeit a tiny one.

00:14:24.480 By extracting this type object and injecting it into the constructor of the columns, we begin to see connections with other behaviors that need adjustment, such as how SQL types are looked up. With the injection of the type object, we must also change surrounding code to accommodate this behavior.

00:15:03.360 We go through our system and replace all these case statements, which are scattered around in the column class. Slowly, we move methods to the type objects. We also introduce a mapping system into our connection adapters to streamline the SQL type string handling, simplifying how we build symbol type representations.

00:16:06.880 At this point, we have refactored our system into something that is easier to override, allowing us to implement the API that will let us interact better with it. The simplest case to start with is changing the type of an attribute from string to integer. We create a model with a schema and two attributes of the same type, then assert that changing the type of one attribute modifies the behavior accordingly. We’ve actually written the first invocation of our API.

00:16:54.760 What we have now is a simple type object that we’ll pass directly to our method. While other alternatives like symbols or constants could be used to denote types, for now, we keep it simple and explicit. This design choice remains consistent through the rest of the refactoring process, simplifying understanding and maintaining clarity in implementation.

00:17:32.720 Implementing the API becomes easier with the object-oriented approach we’ve established. Each object has an interface that communicates its behavior clearly. The organization of methods becomes manageable, and asserting properties becomes straightforward. The API begins to form a recognizable pattern, one that developers can navigate effectively.

00:18:05.840 In this API development, we strived for simplicity but acknowledged the challenges posed by complicated interactions. By streamlining methods and introducing decorators, we encapsulate specific behaviors while ensuring that the user-facing components maintain clarity about their functionalities. We should work against convoluted implementations that obscure the behavior's intent.

00:19:05.720 Expanding on our examples, we illustrate how exterior conditions like timestamps or serialization may also dictate the configuration of types in attributes. We illustrate this approach by examining how we can adapt methods for time zones and serialization while considering the delicate balance required between functionality and simplicity.

00:20:04.240 This exploration emphasizes that adapting modules for simplicity and functionality reflects sound design principles, focusing on the readability, maintainability, and usability of APIs. As we refine our work, we align more closely with the user experience and the needs of developers who will employ these APIs.

00:20:59.880 Continuing our lesson on the Rails system, the significant changes we've undertaken signal a shift towards better handling of typecasting. As we reshape these internals, we draw upon longstanding principles in software design—clarity, coherence, and usability—elements that inform not only how we structure our code, but also how our APIs evolve.

00:21:44.280 By decoupling behaviors, we make future refactoring simpler, and by being mindful of the interfaces we expose, we respect the principle of least astonishment. Moving forward, our focus is on maintaining this balance: how do we enhance functionality without losing sight of clarity? The work continues as we iterate to refine and improve.

00:22:34.160 In moving to Rails 5, these refinements will become crucial. The enhancements ensure that we transition to approaches where attributes and their types are unambiguous, reliable, and coherent, leading to a more robust experience for developers and end-users alike. We anticipate that users will find this more straightforward approach not only improves their coding experience but also enhances overall performance.

00:23:32.600 As the discussion wraps up, it’s evident that clear contracts in code are key. With our design choices, we strive to create expectations that ensure model behaviors remain predictable. Developing these APIs requires diligence, awareness of potential pitfalls, and active engagement with both core features and user necessities.

00:24:23.520 As we conclude, we recognize that the ongoing challenge is to push ourselves to continue evolving both our APIs and implementations. We want to achieve a harmonious blend of flexibility and predictability, where the Rails experience aligns seamlessly with developer expectations. We welcome you to share your insights as we collectively shape the future of Rails.

00:25:14.320 Thank you all for your attention. Are there any questions?

00:25:49.240 Yes, I have used both the 4.1 and 4.2 APIs while making a JDBC adapter, and I prefer your 4.2 API.

00:26:32.079 The question was about the performance impact. While it seems like we have introduced many new objects, it raised some concerns during development. Leading up to RubyConf this year, we got feedback that 4.2 was performing slower than 4.1, and I resigned to fix that.

00:27:18.640 After running tests, I found that while we introduced the attribute objects, we also removed several hashes which resulted in a net performance benefit. Though there may be additional low-hanging performance improvements possible, the overall structure has proved beneficial over raw efficiency.

00:28:03.560 Expectations of performance regressions are often due to overlooked details—such as strings being mutable. We've refined our approach in ways that need further exploration within future iterations.

00:29:01.200 The question was regarding the maintainability of the API by connection adapters. We’ve created a method to look up types for given column objects in Active Record, ensuring that while the adaptation is being refined, interface integrity continues.

00:29:47.560 We don’t want to rewrite all associations unnecessarily, but this adaptability opens doors in utilizing the API effectively under the constraints we’ve set.

00:30:28.320 In responding to the question about changing the model attribute interactions in the future, no such changes are planned, as this would require a deprecation cycle not warranted by the pain involved.

00:31:13.760 Contemplating attributes and custom types reaffirms how important clarity in API design becomes as we evolve and integrate various functionalities.

00:31:55.560 Creating strong internal APIs is a priority, managing the intricacies while supporting meaningful user experiences with our external-facing interfaces.

00:32:37.240 In closing, the refinement of attributes in Rails addresses our long-term goals: ensuring that transitions between states remain clear, functioning correctly regardless of inherent complexities. This journey will require input from our community to continue enhancing the Rails platform.