Software Architecture
Decoupling Persistence (Like There's Some Tomorrow)

Summarized using AI

Decoupling Persistence (Like There's Some Tomorrow)

Piotr Szotkowski • April 06, 2012 • Wrocław, Poland

In the video titled "Decoupling Persistence (Like There's Some Tomorrow)" presented by Piotr Szotkowski at the wroc_love.rb 2012 event, the main theme revolves around the importance of decoupling models from their persistence layers in Ruby on Rails applications. Szotkowski argues that strong coupling between models and databases can lead to architectural pitfalls, as it limits flexibility and complicates testing.

Key points covered in the talk include:
- The Problem with Strong Coupling: Szotkowski articulates that database layers should be implementation details rather than foundational aspects, suggesting that databases are often viewed as black boxes which can complicate development and design.
- Benefits of Decoupling:
- Backend Independence: By decoupling, models can be shifted between different database backends as needed.
- Avoiding Shortcuts: Engaging with persistence from the start can drive developers to make shortcuts that might be detrimental in the long run, especially when switching database engines.
- Improved Testing: Tests can focus on domain logic rather than persistence, leading to cleaner and faster tests.
- Challenges: Szotkowski also acknowledges potential challenges such as performance concerns, perceived complexity for new developers, and the principle of YAGNI (You Aren't Gonna Need It).
- Approaches to Decoupling: Various methods are outlined for achieving separation, including the use of decorators, composition, and the DCI (Data, Context, Interaction) pattern. He emphasizes that separating responsibilities can lead to more maintainable applications.
- Examples and Tools: Szotkowski references tools like NoDB, which helps separate Active Record from the database, and mentions functional patterns that help manage data objects effectively.

In conclusion, Szotkowski encourages the audience to experiment with decoupling persistence and consider alternative paradigms like polyglot persistence. The overarching message is that decoupling can facilitate easier changes to applications and improve overall architecture, resulting in more sustainable and flexible software development practices. The session ends with an invitation for discussions and further exploration of the ideas presented.

Decoupling Persistence (Like There's Some Tomorrow)
Piotr Szotkowski • April 06, 2012 • Wrocław, Poland

This video was recorded on http://wrocloverb.com. You should follow us at https://twitter.com/wrocloverb. See you next year!

From DCI to presenters, from Uncle Bob's architecture talk and Avdi Grimm's upcoming 'Objects on Rails' book to the proliferation of (competing? complementing?) database systems, it seems the time has come to seriously consider decoupling our objects' persistence from the rest of the application.

This talk -- after describing the general vices of strong object/database coupling and the all-too-usual rails g model-driven development -- covers the various approaches to separating the objects' persistence layer, along with their virtues (cleaner, simpler tests! backend independence! no RDBMS-related shortcuts impacting the design!) and potential vices (performance? perceived compexity? YAGNI?).

wroclove.rb 2012

00:00:14.080 Hey, thanks! Welcome. I'm going to talk about decoupling persistence, and mostly that maybe you shouldn't couple your models to your persistence layers. They may not necessarily need to.
00:00:22.080 My name is Piotr. This is my handle.
00:00:27.680 If you don't know what the Medicand University is, you should disregard this talk and just click on the link or Google it and get reading. It's an awesome community that I was able to pass the exam for, and it was the most awesome course that I had.
00:00:33.920 I work at the Wrocław University of Technology. If you remember that handsome guy who was in half of Nick's slides, Alex, here, he got to be an engineer despite me being his supervisor.
00:00:45.360 I also proclaim myself to be the Chief Software Officer at a Warsaw-based consultancy, Ruby Andre's consultancy. I just received a bottle of scotch from one of our clients, so we must be good enough!
00:00:51.680 I also coordinate events for the IT and NGO non-profit organization sector.
00:01:03.520 So how do we create a Rails app or a Ruby web app nowadays? Well, we install the obvious parts, then we install MySQL because we obviously need it.
00:01:10.560 Then we install a JavaScript runtime because that's what you need for a Ruby web application. We don't need to install them because we already have them, and we use them because it's the best editor.
00:01:16.640 If you didn't attend yesterday's Vimcast workshop, you really missed out! There was some scotch there as well.
00:01:22.159 Now, let's say we want to have a website; the print media is getting really large, so we want to create a website for a new paper, 'The Truth.'
00:01:27.439 We create the Rails app, we add the database to 'point.sql,' and we uncomment the Ruby Racer in the Gemfile.
00:01:32.560 Then we generate our first model, our second model, we run 'rake db:migrate,' and then we start editing tests.
00:01:39.600 So this is basically what I think every one of us did at some point. Some of us believe this is how applications should be built.
00:01:45.439 Uncle Bob Martin said that if the first thing you type when writing a Ruby app is 'Rails,' you've already lost the architecture game. You can agree or disagree with Uncle Bob, but I think there's some truth to it.
00:01:58.480 Perhaps we should think about architecture before we even decide on the framework we use.
00:02:06.080 Why are we so obsessed with the database layer? Why do we need databases? We generally believe that our objects are the core of what we build.
00:02:11.599 We want our objects to persist; we want to use them over and over again. We want to be sure they'll be there when we reboot.
00:02:17.440 For some reason, we want this persistence from the start rather than waiting for actual client data.
00:02:22.800 So, what's the problem with databases? The database is a black box to which you put your data.
00:02:28.879 If you're lucky, you'll get it back again, and if you're very lucky, you'll get it back in a usable form.
00:02:36.480 Databases for persistence should be an implementation detail, and that’s one notion you might or might not subscribe to.
00:02:42.959 It doesn’t necessarily have to be the first thing you use. There are quirks with certain database systems, and they can complicate matters.
00:02:50.560 It looks like you're creating a database, most likely with Latin-1 encoding and Swedish collation.
00:02:58.480 Maybe we shouldn't bother with databases so early in the process.
00:03:06.080 There are various database systems to choose from, such as MySQL, PostgreSQL, and SQLite for small apps.
00:03:11.599 For example, SQLite has an embedded version, and CouchDB, MongoDB, and other document databases provide embedded versions, too.
00:03:20.720 We can even run our tests against embedded versions.
00:03:27.280 We have various databases, such as in-memory Redis, or a Redis object collection. If you don't know about it, you should try Ruby objects with Redis data.
00:03:32.479 There are graph databases for social networks and proximity searches, and there are directory services for managing logins across different web apps.
00:03:39.199 LDAP is one approach to look into. There's also the Young Record for Active Record or Active Model-like approaches for YAML files.
00:03:46.000 PStore, a standard Ruby library, provides persistence for objects for free. If you're writing a small app that you may be the sole user of, consider PStore.
00:03:53.919 There are also object databases; MagLev has been released. If you're unfamiliar with MagLev, it’s a Ruby VM built on a Smalltalk-like framework.
00:03:59.919 It has its own object persistence, and Madeleine is a great project for persisting objects in a Java-like provider manner.
00:04:06.960 There’s also QB Hole, a lovely project that persists your objects in a Unix-like standard.
00:04:13.440 Martin Fowler mentioned that now, with more people considering NoSQL, more may consider no database. Persisting might not be the first thing you should focus on.
00:04:20.240 Even if you want to persist data early, you should think about where you wish to do so.
00:04:27.439 There's a wonderful presentation by Martin Fowler that I shamelessly referenced, covering various database systems across different parts of your application.
00:04:35.280 The idea of polyglot persistence is that a lot of data can be stored in very particular kinds of data storage.
00:04:43.120 Maybe you shouldn't decide upfront which one you're going to use for all your data.
00:04:49.520 So, what does splitting models and persistence give us?
00:04:56.800 One quote I found relevant to this presentation is that when I open up Rails projects, they are often filled with 100 Active Record classes.
00:05:01.360 Each model should not equal persistence. We need something better.
00:05:06.800 Persistence and logic are two separate responsibilities that every Rails app combines.
00:05:13.440 We shouldn't take for granted that our models will persist. We might think we have a simple application that runs on MySQL.
00:05:21.440 We might assume Active Record is the best, but that’s not always the case.
00:05:27.600 So what are the benefits of splitting models? First, backend independence. You can move some of your models to one backend and some to a different backend.
00:05:35.040 Secondly, no shortcuts. If you persist right from the start, you sometimes make shortcuts.
00:05:41.440 You might take known database shortcuts. When switching database engines, you'll discover the shortcuts you relied upon.
00:05:48.560 On the other hand, explicit shortcuts can benefit you. For example, PostgreSQL has a specific data type for geolocation.
00:05:55.680 You can instruct the database to provide you with people within a five-kilometer radius around specific coordinates.
00:06:03.360 If you implement that in your model layer without thinking about persistence, when you switch engines, you'll have to re-implement geolocation.
00:06:09.760 Arguably, especially if you read 'Objects on Rails' from Avdi Grimm, it's easier to model your objects when you don't constantly consider persistence.
00:06:17.919 It's also more future-proof; switching persistence layers is easier if it is a separate layer.
00:06:23.919 Tests are cleaner and simpler. They actually test your domain objects—not the persistence—and they run extremely fast.
00:06:31.840 It's wonderful to avoid raising the entire Rails stack just to test your models. You test the persistence separately.
00:06:39.360 If persistence is merely a thin driver that maps your model to a database, it's very easy to test.
00:06:46.160 However, when it grows, it's still better to test it separately rather than relying entirely on unit tests.
00:06:53.120 Integration tests are important, but you shouldn't necessarily unit test persistence.
00:07:01.120 Rice is not your application; don’t unit test it. If your models inherit from Active Record, you’re essentially testing Active Record.
00:07:07.440 You’ll also encounter all sorts of finding protocols that become hard to test if you try to do so extensively.
00:07:15.200 So, what could be the potential criticisms of this approach? Performance should be considered.
00:07:23.600 You should think about it upfront, especially if abstraction may become a performance liability.
00:07:30.720 Perceived complexity could also arise. It's easier to work with Active Record models, as they are familiar.
00:07:37.600 But if you model your application correctly, maybe separating persistence isn't as large of an overhead for a new developer.
00:07:45.440 You ain't gonna need it! That’s a serious question—should you even focus on it?
00:07:52.000 However, you usually have a vague idea of whether your app is going to be a big one or just a small project.
00:08:00.480 NoDB is a great library from AppDream that separates your Active Record from the database.
00:08:07.200 Try it out! If you don't want a separate persistence layer but want faster, less brittle tests.
00:08:14.240 The primary argument for this approach is that it's object-oriented. Your domain models should have clean-cut responsibilities.
00:08:21.440 Persisting themselves might not be the responsibility you want to assign to each one.
00:08:30.560 The biggest reason to decouple is to make our applications easier to change.
00:08:39.600 The only constant in life is change, and this is especially true for software projects.
00:08:46.720 Rails is reaching a point where lots of projects are starting to mature, yet many developers feel their projects are not as easy to modify as they used to be.
00:08:54.560 This separation can help facilitate future changes. Attempts to predict which parts of your codebase will need those changes have often ended badly.
00:09:02.639 Changes are often unpredictable, much as premature optimization tends to miss the mark.
00:09:09.680 Consider whether a lightweight persistence layer will make it easier for your projects to change. Software projects evolve more than we realize.
00:09:13.680 Thinking about a project you worked on a year ago, it likely looks different now.
00:09:20.000 So how do we approach separation? It’s a big topic with various methods for moving persistence away from primary models. The idea is to extract as much logic as possible out of data objects.
00:09:39.280 Data objects are things you process; that's one way to achieve it. Creators are a good example.
00:09:46.079 If you have JSON or XML methods within your models, they represent themselves instead of relying on external representations.
00:09:54.560 Maybe this responsibility should be offloaded to decorators or similar patterns.
00:10:02.879 Composition is another interesting method; your domain objects could be composed of persistence components.
00:10:10.560 The final example is called DCI, which stands for Data, Context, and Interaction. I won’t go into detail as I'm still wrapping my head around it.
00:10:17.680 A quick explanation of decorators is simple. If you want to remove a part of your domain object's responsibility, using a decorator can be convenient.
00:10:23.200 Draper is a great resource for decorators. I’ll provide a simple example using the library SimpleDelegator, which wraps the given object and allows for easy decoration.
00:10:29.840 For instance, we could have an Address class; it could be either persisted or just modeled.
00:10:35.760 We can give it an Active Record-like API—let's say it's initialized with a hash.
00:10:41.680 We want to display the address in a specific format, which may differ by region.
00:10:48.480 A Polish address is usually displayed as street, postcode, city, and country on a separate line.
00:10:54.880 We might want a PolishAddress class that decorates the Address class with a formatted method calling street, postcode, city, and country methods.
00:11:01.600 To achieve this, we inherit from SimpleDelegator and call super with the object.
00:11:07.760 The PolishAddress class will then have access to all methods of the base object.
00:11:14.479 This way a decorator can assist in restructuring domain data without compromise.
00:11:22.879 The benefit is ease of testing; you can inject any object, ensuring functionality and reuse when applicable.
00:11:30.560 Another method is composition over inheritance. There is a quote, "Inheritance is code smell." I don’t fully agree.
00:11:37.440 However, we should indeed look out for inappropriate inheritance, especially in specific codebases.
00:11:44.000 Maybe your domain objects do not need to inherit from anything. Instead, they can be composed of a persistence part and a functional part.
00:11:51.679 A simple example from Ruby’s standard library presents the Forwardable module.
00:11:59.280 Consider we have an Address object and a Person object with given names and surnames, it’s vital to recognize naming variations based on cultural context.
00:12:07.360 To manage this complexity, we can create Address and Person domain events for managing rentals.
00:12:15.680 We will then create a decorator that drives the formatting, synthesizing the address attributes.
00:12:23.040 The Polish Envelope Address illustrates this concept effectively. By forming various delegators, we can concatenate functions easily.
00:12:30.720 Ultimately, we need to consider splitting our data models and responsibilities, maintaining separation to safeguard functional conditions and grow applications.
00:12:38.480 Next, let's briefly discuss DCI. DCI means Data, Context, and Interaction; central to this is the idea of separating behavior.
00:12:46.000 Data objects hold only data, and it's critical to make clear distinctions among various roles.
00:12:54.720 You could have a landlord and tenant who both share attributes but differ based on context.
00:13:01.680 Your roles extend via initializing the context to encompass defining responsibilities.
00:13:08.720 Upon executing the context, roles and responsibilities become apparent and are contextually managed.
00:13:17.040 A foundational example might be to rent a place to a tenant secured under the landlord’s role.
00:13:26.000 The layer of behavior should then enhance the clarity of how the system operates.
00:13:36.320 The last point to discuss, which might be slightly overboard, is treating web apps as sole objects while hiding persistence.
00:13:44.560 There are remarkable examples such as 'Candy' and 'Ambition,' which encapsulate persistent capabilities in a seamless manner.
00:13:53.520 Candy demonstrates a way to use transparent persistence in MongoDB. You can create a conference class and include it.
00:14:02.560 It automatically connects to localhost and manages everything involved in creating objects, with only a few lines of code.
00:14:10.880 Another interesting approach involves Ambition, which allows Ruby-like syntax for persistent queries without revealing underlying technologies.
00:14:18.160 You can write readable queries, enabling you to translate them seamlessly to SQL or other structures.
00:14:25.040 Both approaches facilitate efficient programming experiences and enhance flexibility.
00:14:33.440 They challenge conventional methods of persistence and encourage fresh perspectives.
00:14:42.720 I encourage you to play around and discover innovative ways to navigate persistence challenges.
00:14:51.440 Consider whether to decouple models from persistence and explore alternatives.
00:14:58.720 Learn from the features of various persistence layers.
00:15:05.600 PostgreSQL, for example, supports array columns useful for tagging.
00:15:12.160 Reflect on the concepts of creators, composition, DCI, or any ideas that arise.
00:15:18.720 Please check out 'Personal Names Around the World.' That's the second funny picture I keep using, representing us engineers.
00:15:26.560 Let's play with these concepts, experiment, and have fun!
00:15:48.960 The questions will stand between you and lunch, so I will take the blame, but if you have any, please go ahead!
00:15:54.799 If not, you can find me afterwards. Skip your next two coffees and buy a five-dollar book from Avdi Grimm, which is absolutely awesome!
00:16:03.158 Watch Uncle Bob's talk; you might disagree with it, but do watch it. There’s a great article by Piotr Sonita about making Active Record models thinner.
00:16:11.440 Check out the presentation from Martin Fowler, a great Ruby Rocks podcast, and there’s a hackathon next weekend in Warsaw that I organize.
00:16:19.440 If you want to come and help a non-profit organization to create amazing projects, we had one in December that was fantastic.
00:16:26.880 I'll be around; I'm wearing red like Santa Claus to hide the bloodstains.
00:16:34.560 Please find me, and enjoy the party!
Explore all talks recorded at wroclove.rb 2012
+11