00:00:15.379
Thank you for coming. The title of this talk is 'Building Generic Software.' My name is Chris Salzberg, and I have a lot to discuss today.
00:00:27.660
I will cover the easier parts quickly and delve into the more challenging aspects slowly. A few quick points about myself: my handle is 'Chewy,' which you may have seen around.
00:00:41.370
I live in Tokyo, Japan, as you can see in the background of this slide. This is Tokyo. Don’t be fooled; I’m not Japanese. That’s just my icebreaker joke. I’m actually Canadian, originally from Montreal. I work at a company called 'Dajika,' based in Tokyo, and I contribute to the open-source world.
00:00:58.199
I’m the author of several Ruby gems, the most well-known of which is called 'Mobility.' I will talk about that a bit later. In my free time, I write about topics like the module builder pattern, which you may have heard of.
00:01:10.439
The focus of today's talk is building generic software. To kick things off, I will introduce the concept of generic software: What is it, and why should you care? Based on this idea, we will look for a specific problem and use it to build some generic software, specifically a framework.
00:01:27.420
After that, we will see what lessons we can learn from this exercise. Now, what is generic software? I did not invent this term, as far as I know. It was coined by Jeremy Evans, who gave a talk here yesterday. He’s the author of a well-known CRM called 'Sequel,' similar to Active Record, and another well-known gem called 'Rhoda,' which he actually discussed here in 2014.
00:01:46.110
I’ve been following Jeremy’s work for some time. A while ago, I sought inspiration in his open-source projects and found a quote buried in his 2012 slides from a talk on the development of Sequel. In this quote, he states that one of the best ways to write flexible software is to write generic software. Instead of designing a single API that completely handles a specific case, you write multiple APIs that manage smaller, more generic parts of that use case. Then, handling the entire case involves gluing those parts together. When you approach code this way, designing APIs that solve generic problems ensures that you can more easily reuse those APIs later to solve other issues. This quote resonated with me and aligned with the work I was doing on developing generic components.
00:02:18.010
That's why I'm here today to make this talk. As I delved into this idea, I realized that you can take the general concept of building blocks and take it in several different directions, depending on how you apply it. This tweet from Gary Bernhardt from September encapsulates that. You may know him as a Rubyist; he stated that programming should be like assembling Lego. Decompose everything into tiny pieces with maximally general interfaces.
00:02:36.129
This idea aligns closely with Jeremy Evans's quote. While you can take my word for it, I'd suggest you listen to Jeremy—he's done great work! However, moving ahead a decade, simple tests have become increasingly complicated, and at times, they break for reasons no one understands. This brings us to a strange schism. Depending on how you apply this Lego block philosophy, you either end up with maintainable software or a convoluted mess that's challenging to maintain.
00:03:05.620
Here’s another perspective: a comment on a Hacker News thread responding to a blog post that discussed how our software has become larger and more complex while doing the same tasks. The commenter attributes part of the blame to an unhealthy obsession with code reusability and sticking to specific paradigms. The result? There’s so much code comprising abstractions piled upon abstractions that no one seems to just want to write some straightforward code anymore; they’d prefer to include entire frameworks instead of writing a few helper functions.
00:03:29.680
I sympathize with this sentiment. If you look at the Twitter thread Gary Bernhardt participated in, it echoes sentiments suggesting that you should just write the code yourself rather than forcing these libraries upon yourself if you don’t genuinely need them. While that's reasonable, I think it misses a crucial point. Imagine that this tree represents a giant dependency tree, like Ruby gems. Most of the time, we’re at the leaves of this tree, building applications that aren't required by anyone else. We rely heavily on lots of software that branches down to the roots, ultimately to the Ruby interpreter. So, suggesting we rip out these frameworks is akin to chopping off the branches from our leaves.
00:03:59.919
While that is a valid approach, I believe we should also consider how we can improve this tree. Can we enhance these branches? Can we optimize and strengthen this structure? That’s what I want to discuss today. To do this, I’ll utilize some concepts that date back over 30 years, back to the 1980s, before even the Internet. I want to highlight a significant paper from 1988 written by Ralph Johnson and Brian Foote, who you may know if you've encountered the term ‘Big Ball of Mud.’ The title is 'Designing Reusable Classes.'
00:04:29.320
This entire idea of reusability is fundamental to object-oriented programming, concerning key concepts like inheritance and polymorphism, which are designed to promote code reuse. Additionally, the concept of a framework, which I will discuss later, is introduced in this paper. Today, I intend to take some of these timeless ideas and apply them alongside the notion of generic software to guide our journey towards creating maintainable software.
00:04:50.610
Generally, when you want to build generic software—software that is reusable and applicable in multiple contexts—you have to start with something specific, at least in my experience. You don't begin from the bottom of the tree; you start from the leaves and work downwards. Therefore, I will begin with a specific example: translation. This topic is particularly relevant to me, as my entry point into the Ruby community was through a translation platform.
00:05:09.430
So, let’s explore the concept of translated attributes. It’s actually a straightforward idea. I’m using the I18n locale here, and it’s not crucial, but take a global language, like English. You might have an attribute in a class, say a 'Talk' class. You create an instance of 'Talk' and set your title.
00:05:27.280
For instance, if you set the title to 'Building Generic Software,' you would expect to retrieve that title later. However, if we switch the locale to something like Japanese and fetch the title, we get nil because, at this moment, there's no translation for that attribute in Japanese. To resolve this, we would need to add that translation, and subsequently, upon switching back to English, the correct English translation would be retrieved.
00:05:42.570
And that’s essentially how it works. In defining these translations, there are various Ruby gems constantly emerging that manage translated attributes. They each have their conventions, but generally, they feature a common interface: you call some class method, often referred to as 'translates,' and pass in your attribute names. This dynamically creates translated attributes for you to use.
00:06:00.740
The complexity arises, however, when we consider how to store these translated attributes. Here is where we start to explore the idea of generic software moving forward. So, what do these storage patterns look like? The simplest approach is the idea of translatable columns, which is pretty straightforward. Imagine a 'Comment' model with a corresponding 'comments' table.
00:06:24.470
If you call the ‘content’ method by default, your ORM—whether it's Sequel or Active Record—won’t understand what you’re referencing because there’s no column named 'content' in your structure. Instead, you would create separate columns for every language you want to support translations for. For example, you’d have 'content_en' for English, 'content_fr' for French, and so forth. You'd need to dynamically define a method that maps from the 'content' call to the respective translated column based on the current language.
00:06:43.750
This approach is relatively transparent and works well, but it has its downsides such as requiring migrations every time a new language is added. Another more scalable approach involves using translation tables. Instead of storing translations in the model table, you create a separate translation table.
00:07:03.480
In this model, each translation is stored on this separate table, with the language represented as a new column rather than a suffix on a model column name. You can set up a foreign key reference back to your original comment table, and then for each translation, you have the relevant column name to store those translations.
00:07:22.720
You would create an association, and when you want to fetch a translation, you'd access this association and retrieve the relevant translation based on the current language. Additionally, there are various patterns to implement storage as well. For instance, you could also use JSON columns in Postgres or the more recent versions of MySQL which also supports JSON.
00:07:40.440
By doing this, you can place all the translations into a single column where the keys are the language codes and the values represent the translations. These storage patterns form a foundational layer, and different gems implement various access patterns on top of them.
00:08:02.660
The most common access pattern is fallbacks, especially if you've worked with I18n. This approach enables you to set a primary language and a fallback language, so if you try to fetch a translation in a regional dialect like Canadian English (en-CA) without a specific entry, you could fall back to the more general English translation.
00:08:20.950
Implementing fallbacks involves looping through the locales you’re trying to access, trying one language at a time, and returning the first found translation. It's quite a simple logic. The important part about design in this context is being able to support various patterns while not coupling them together too heavily.
00:08:42.540
You could also implement other features like dirty tracking, which allows you to track changes in active records. Unfortunately, when managing translated attributes, your ORM may not automatically support this because it doesn’t recognize these custom attributes without additional magic.
00:08:58.990
So, if I set a title to 'Building Generic Software' and change it to 'Building Specific Software,' how do you track those changes? There are additional challenges when you want to perform lookups based on translations in a different language. These aspects need special handling during implementation.
00:09:23.200
Now, let’s delve into the most intriguing part: how different gems can combine various functionalities using control flow. Typically, I illustrate this with the ‘translates’ method. This method defines the accessor for translated attributes.
00:09:49.440
The process begins by splatting the attributes passed into it and iterating through each attribute. You would call a method to define an accessor for each attribute. This would generally involve defining methods for getting and setting the translation values, but the implementation can vary depending on which gem you’re utilizing.
00:10:14.610
The internal details may change, but in essence, this defines how you store and retrieve translations. Now, as a concrete example, if we have a 'Talk' class and the attribute we're interested in is 'title,' the process will result in methods that read from storage and write to storage.
00:10:32.760
The key point here is identifying what these methods do. This explanation is adapted from a gem called 'Tracker,' which is designed to handle translatable columns. Here’s how it typically works. First, we call the necessary fallbacks and attempt to access the value. Since you might be using a storage pattern, you will refer to the required column translation and manage the relevant logic accordingly.
00:10:50.720
So, as you can see, this process includes the handling of various situations seamlessly. If the translation exists, great! If not, it can continue through the list of fallbacks until it finds a suitable resource.
00:11:07.000
Now, we discussed a problem earlier and how we can address it. The issue in question involves a gem called Globalize, which you might know—it’s the most recognized gem for managing translations in Rails applications. The repository has been active, and one of the authors, Thomas, posted a thought-provoking issue about two and a half years ago.
00:11:28.600
At that time, the Rails community began to embrace Postgres, which introduced features like JSON storage and JSONB. Thomas considered whether it might be feasible to allow a method to switch from a storage table-based approach to JSON storage or JSONB storage, indicating an interest in making the setup more flexible.
00:11:46.460
This idea provoked my interest as I began to ponder what steps would be necessary to make a system pluggable. Thus, the critical question for us is: how do we achieve flexibility in such a way that different approaches can coexist?
00:12:01.740
Going back to our theoretical model we’ve built, we need a way to allow for extensibility. The high-level application code calls method definitions, like determines how it accesses translated attributes. Initially, we would pass a class method into our translates method to build this architecture.
00:12:17.760
While embedding specific storage references, we aim to peel the functionalities apart so that we can replace them with alternate implementations as needed.
00:12:34.600
Instead of directly accessing an inner method, we elevate this responsibility to the application layer. This notion of 'inversion of control' is crucial here, where we want the high-level code to dictate how functionalities tie together.
00:12:48.700
In designing reusable classes, as per the paper I mentioned earlier from 1988, this idea allows frameworks to extend their base implementations while promoting a flexible and collaborative design approach.
00:13:06.950
The concept is catchy, often referred to as the 'Hollywood principle'—don’t call us; we’ll call you. You design your plugins or modules, and when the main program needs behavior, it calls back to the plugins, allowing for divergence in functionality without coupling all the pieces tightly.
00:13:23.510
We can now proceed to build a second version of the API. This new approach can accommodate multiple types of storage while ensuring the access methods remain flexible. The 'translates' method can take additional parameters, such as backend keyword arguments representing the storage strategy.
00:13:43.300
As we implement these changes, it’s essential to ensure that whatever backend class you pass also adheres to specific expected behaviors or protocols. This offers a superior level of flexibility, as now you can implement a backend that accommodates existing standards and allows others to follow suit.
00:14:02.260
Once this method is defined, you would manage the attributes accordingly, going through each input while keeping track of instance variables. These backend classes should avoid unnecessary complexities, merely defining how the data reads and writes.
00:14:20.580
After defining the backend method, you can create a way to dynamically access your storage strategy without hard coding the references. This flexibility is crucial for achieving the goals of generic software.
00:14:35.030
As I mentioned earlier, if we hypothetically replace the original backend functions with a translation table backend, we must remember that the core mechanism of accessing translations remains unmoved.
00:14:55.130
The implementation must properly process the references without requiring any hardcoded definitions in the model, which enables seamless integration without necessitating reworks or dependencies.
00:15:10.370
Along the way, we determined that a 'setup model' function can be a valuable mechanism. Invoking this would provide an entry point for configuration about the nature of the backend without disturbing the core setup. Therefore, both the backend and the access patterns do not interfere with the direct usage of either.
00:15:30.470
This research into creating a more flexible approach yields a model wherein distinctions among backend functionality and features are maintained. Extensive interactions enable each part to uphold its independent yet integrated function.
00:15:45.830
Ultimately, when fully realized, you find the backend protocols allowing for different storage methods to work harmoniously, creating an ecosystem where you can cater to various data-access needs.
00:16:07.070
The beauty lies in how different components come together without forcing external dependencies to become interconnected.
00:16:23.140
At this point, I want to reference some notable gems that adopt similar philosophies, notably 'Rhoda' and 'Shrine.' They both have strong modular designs where one can engage with various plugins or storage points while harnessing the Ruby ecosystem’s flexibility.
00:16:39.130
In sharing these insights, I hope to illuminate not only my own journey but also the expansive potential of norms around developing interest-generating software in the Ruby community.
00:16:56.240
What sets successful frameworks apart is their pluggability. Understanding how design patterns contribute to the overall structure aids the development of a combination of reusable and extensible systems.
00:17:12.370
Looking back at the earlier discussion, we contemplated the potential pitfalls of generic software development and specified pathways to mitigation by detailed analysis of our frameworks.
00:17:28.000
This dual-axis reflection—understanding reusability and recognizing complexity—is essential in navigating contemporary architecture. We must strive for balance in these pursuits to prevent our solutions from becoming overly convoluted.
00:17:45.320
As we progress in this discussion, I envision a world where we build exacting tools that deliver generative functions initially and evolve into multifaceted solutions as enhancements and ideas come forth.
00:18:02.650
I urge you to delve into this topic with me if this resonates with you; let's collaborate to further our ecosystem by cultivating well-designed, reusable software.
00:18:28.830
Thank you all for your attention.
