Talks
Decoupling Persistence (Like There's Some Tomorrow)
Summarized using AI
Decoupling Persistence (Like There's Some Tomorrow)
by Piotr SzotkowskiIn the video titled "Decoupling Persistence (Like There's Some Tomorrow)" presented by Piotr Szotkowski at the wroc_love.rb 2012 event, the main theme revolves around the importance of decoupling models from their persistence layers in Ruby on Rails applications. Szotkowski argues that strong coupling between models and databases can lead to architectural pitfalls, as it limits flexibility and complicates testing.
Key points covered in the talk include:
- The Problem with Strong Coupling: Szotkowski articulates that database layers should be implementation details rather than foundational aspects, suggesting that databases are often viewed as black boxes which can complicate development and design.
- Benefits of Decoupling:
- Backend Independence: By decoupling, models can be shifted between different database backends as needed.
- Avoiding Shortcuts: Engaging with persistence from the start can drive developers to make shortcuts that might be detrimental in the long run, especially when switching database engines.
- Improved Testing: Tests can focus on domain logic rather than persistence, leading to cleaner and faster tests.
- Challenges: Szotkowski also acknowledges potential challenges such as performance concerns, perceived complexity for new developers, and the principle of YAGNI (You Aren't Gonna Need It).
- Approaches to Decoupling: Various methods are outlined for achieving separation, including the use of decorators, composition, and the DCI (Data, Context, Interaction) pattern. He emphasizes that separating responsibilities can lead to more maintainable applications.
- Examples and Tools: Szotkowski references tools like NoDB, which helps separate Active Record from the database, and mentions functional patterns that help manage data objects effectively.
In conclusion, Szotkowski encourages the audience to experiment with decoupling persistence and consider alternative paradigms like polyglot persistence. The overarching message is that decoupling can facilitate easier changes to applications and improve overall architecture, resulting in more sustainable and flexible software development practices. The session ends with an invitation for discussions and further exploration of the ideas presented.
00:00:14.080
Hey, thanks! Welcome. I'm going to talk about decoupling persistence, and mostly that maybe you shouldn't couple your models to your persistence layers. They may not necessarily need to.
00:00:22.080
My name is Piotr. This is my handle.
00:00:27.680
If you don't know what the Medicand University is, you should disregard this talk and just click on the link or Google it and get reading. It's an awesome community that I was able to pass the exam for, and it was the most awesome course that I had.
00:00:33.920
I work at the Wrocław University of Technology. If you remember that handsome guy who was in half of Nick's slides, Alex, here, he got to be an engineer despite me being his supervisor.
00:00:45.360
I also proclaim myself to be the Chief Software Officer at a Warsaw-based consultancy, Ruby Andre's consultancy. I just received a bottle of scotch from one of our clients, so we must be good enough!
00:00:51.680
I also coordinate events for the IT and NGO non-profit organization sector.
00:01:03.520
So how do we create a Rails app or a Ruby web app nowadays? Well, we install the obvious parts, then we install MySQL because we obviously need it.
00:01:10.560
Then we install a JavaScript runtime because that's what you need for a Ruby web application. We don't need to install them because we already have them, and we use them because it's the best editor.
00:01:16.640
If you didn't attend yesterday's Vimcast workshop, you really missed out! There was some scotch there as well.
00:01:22.159
Now, let's say we want to have a website; the print media is getting really large, so we want to create a website for a new paper, 'The Truth.'
00:01:27.439
We create the Rails app, we add the database to 'point.sql,' and we uncomment the Ruby Racer in the Gemfile.
00:01:32.560
Then we generate our first model, our second model, we run 'rake db:migrate,' and then we start editing tests.
00:01:39.600
So this is basically what I think every one of us did at some point. Some of us believe this is how applications should be built.
00:01:45.439
Uncle Bob Martin said that if the first thing you type when writing a Ruby app is 'Rails,' you've already lost the architecture game. You can agree or disagree with Uncle Bob, but I think there's some truth to it.
00:01:58.480
Perhaps we should think about architecture before we even decide on the framework we use.
00:02:06.080
Why are we so obsessed with the database layer? Why do we need databases? We generally believe that our objects are the core of what we build.
00:02:11.599
We want our objects to persist; we want to use them over and over again. We want to be sure they'll be there when we reboot.
00:02:17.440
For some reason, we want this persistence from the start rather than waiting for actual client data.
00:02:22.800
So, what's the problem with databases? The database is a black box to which you put your data.
00:02:28.879
If you're lucky, you'll get it back again, and if you're very lucky, you'll get it back in a usable form.
00:02:36.480
Databases for persistence should be an implementation detail, and that’s one notion you might or might not subscribe to.
00:02:42.959
It doesn’t necessarily have to be the first thing you use. There are quirks with certain database systems, and they can complicate matters.
00:02:50.560
It looks like you're creating a database, most likely with Latin-1 encoding and Swedish collation.
00:02:58.480
Maybe we shouldn't bother with databases so early in the process.
00:03:06.080
There are various database systems to choose from, such as MySQL, PostgreSQL, and SQLite for small apps.
00:03:11.599
For example, SQLite has an embedded version, and CouchDB, MongoDB, and other document databases provide embedded versions, too.
00:03:20.720
We can even run our tests against embedded versions.
00:03:27.280
We have various databases, such as in-memory Redis, or a Redis object collection. If you don't know about it, you should try Ruby objects with Redis data.
00:03:32.479
There are graph databases for social networks and proximity searches, and there are directory services for managing logins across different web apps.
00:03:39.199
LDAP is one approach to look into. There's also the Young Record for Active Record or Active Model-like approaches for YAML files.
00:03:46.000
PStore, a standard Ruby library, provides persistence for objects for free. If you're writing a small app that you may be the sole user of, consider PStore.
00:03:53.919
There are also object databases; MagLev has been released. If you're unfamiliar with MagLev, it’s a Ruby VM built on a Smalltalk-like framework.
00:03:59.919
It has its own object persistence, and Madeleine is a great project for persisting objects in a Java-like provider manner.
00:04:06.960
There’s also QB Hole, a lovely project that persists your objects in a Unix-like standard.
00:04:13.440
Martin Fowler mentioned that now, with more people considering NoSQL, more may consider no database. Persisting might not be the first thing you should focus on.
00:04:20.240
Even if you want to persist data early, you should think about where you wish to do so.
00:04:27.439
There's a wonderful presentation by Martin Fowler that I shamelessly referenced, covering various database systems across different parts of your application.
00:04:35.280
The idea of polyglot persistence is that a lot of data can be stored in very particular kinds of data storage.
00:04:43.120
Maybe you shouldn't decide upfront which one you're going to use for all your data.
00:04:49.520
So, what does splitting models and persistence give us?
00:04:56.800
One quote I found relevant to this presentation is that when I open up Rails projects, they are often filled with 100 Active Record classes.
00:05:01.360
Each model should not equal persistence. We need something better.
00:05:06.800
Persistence and logic are two separate responsibilities that every Rails app combines.
00:05:13.440
We shouldn't take for granted that our models will persist. We might think we have a simple application that runs on MySQL.
00:05:21.440
We might assume Active Record is the best, but that’s not always the case.
00:05:27.600
So what are the benefits of splitting models? First, backend independence. You can move some of your models to one backend and some to a different backend.
00:05:35.040
Secondly, no shortcuts. If you persist right from the start, you sometimes make shortcuts.
00:05:41.440
You might take known database shortcuts. When switching database engines, you'll discover the shortcuts you relied upon.
00:05:48.560
On the other hand, explicit shortcuts can benefit you. For example, PostgreSQL has a specific data type for geolocation.
00:05:55.680
You can instruct the database to provide you with people within a five-kilometer radius around specific coordinates.
00:06:03.360
If you implement that in your model layer without thinking about persistence, when you switch engines, you'll have to re-implement geolocation.
00:06:09.760
Arguably, especially if you read 'Objects on Rails' from Avdi Grimm, it's easier to model your objects when you don't constantly consider persistence.
00:06:17.919
It's also more future-proof; switching persistence layers is easier if it is a separate layer.
00:06:23.919
Tests are cleaner and simpler. They actually test your domain objects—not the persistence—and they run extremely fast.
00:06:31.840
It's wonderful to avoid raising the entire Rails stack just to test your models. You test the persistence separately.
00:06:39.360
If persistence is merely a thin driver that maps your model to a database, it's very easy to test.
00:06:46.160
However, when it grows, it's still better to test it separately rather than relying entirely on unit tests.
00:06:53.120
Integration tests are important, but you shouldn't necessarily unit test persistence.
00:07:01.120
Rice is not your application; don’t unit test it. If your models inherit from Active Record, you’re essentially testing Active Record.
00:07:07.440
You’ll also encounter all sorts of finding protocols that become hard to test if you try to do so extensively.
00:07:15.200
So, what could be the potential criticisms of this approach? Performance should be considered.
00:07:23.600
You should think about it upfront, especially if abstraction may become a performance liability.
00:07:30.720
Perceived complexity could also arise. It's easier to work with Active Record models, as they are familiar.
00:07:37.600
But if you model your application correctly, maybe separating persistence isn't as large of an overhead for a new developer.
00:07:45.440
You ain't gonna need it! That’s a serious question—should you even focus on it?
00:07:52.000
However, you usually have a vague idea of whether your app is going to be a big one or just a small project.
00:08:00.480
NoDB is a great library from AppDream that separates your Active Record from the database.
00:08:07.200
Try it out! If you don't want a separate persistence layer but want faster, less brittle tests.
00:08:14.240
The primary argument for this approach is that it's object-oriented. Your domain models should have clean-cut responsibilities.
00:08:21.440
Persisting themselves might not be the responsibility you want to assign to each one.
00:08:30.560
The biggest reason to decouple is to make our applications easier to change.
00:08:39.600
The only constant in life is change, and this is especially true for software projects.
00:08:46.720
Rails is reaching a point where lots of projects are starting to mature, yet many developers feel their projects are not as easy to modify as they used to be.
00:08:54.560
This separation can help facilitate future changes. Attempts to predict which parts of your codebase will need those changes have often ended badly.
00:09:02.639
Changes are often unpredictable, much as premature optimization tends to miss the mark.
00:09:09.680
Consider whether a lightweight persistence layer will make it easier for your projects to change. Software projects evolve more than we realize.
00:09:13.680
Thinking about a project you worked on a year ago, it likely looks different now.
00:09:20.000
So how do we approach separation? It’s a big topic with various methods for moving persistence away from primary models. The idea is to extract as much logic as possible out of data objects.
00:09:39.280
Data objects are things you process; that's one way to achieve it. Creators are a good example.
00:09:46.079
If you have JSON or XML methods within your models, they represent themselves instead of relying on external representations.
00:09:54.560
Maybe this responsibility should be offloaded to decorators or similar patterns.
00:10:02.879
Composition is another interesting method; your domain objects could be composed of persistence components.
00:10:10.560
The final example is called DCI, which stands for Data, Context, and Interaction. I won’t go into detail as I'm still wrapping my head around it.
00:10:17.680
A quick explanation of decorators is simple. If you want to remove a part of your domain object's responsibility, using a decorator can be convenient.
00:10:23.200
Draper is a great resource for decorators. I’ll provide a simple example using the library SimpleDelegator, which wraps the given object and allows for easy decoration.
00:10:29.840
For instance, we could have an Address class; it could be either persisted or just modeled.
00:10:35.760
We can give it an Active Record-like API—let's say it's initialized with a hash.
00:10:41.680
We want to display the address in a specific format, which may differ by region.
00:10:48.480
A Polish address is usually displayed as street, postcode, city, and country on a separate line.
00:10:54.880
We might want a PolishAddress class that decorates the Address class with a formatted method calling street, postcode, city, and country methods.
00:11:01.600
To achieve this, we inherit from SimpleDelegator and call super with the object.
00:11:07.760
The PolishAddress class will then have access to all methods of the base object.
00:11:14.479
This way a decorator can assist in restructuring domain data without compromise.
00:11:22.879
The benefit is ease of testing; you can inject any object, ensuring functionality and reuse when applicable.
00:11:30.560
Another method is composition over inheritance. There is a quote, "Inheritance is code smell." I don’t fully agree.
00:11:37.440
However, we should indeed look out for inappropriate inheritance, especially in specific codebases.
00:11:44.000
Maybe your domain objects do not need to inherit from anything. Instead, they can be composed of a persistence part and a functional part.
00:11:51.679
A simple example from Ruby’s standard library presents the Forwardable module.
00:11:59.280
Consider we have an Address object and a Person object with given names and surnames, it’s vital to recognize naming variations based on cultural context.
00:12:07.360
To manage this complexity, we can create Address and Person domain events for managing rentals.
00:12:15.680
We will then create a decorator that drives the formatting, synthesizing the address attributes.
00:12:23.040
The Polish Envelope Address illustrates this concept effectively. By forming various delegators, we can concatenate functions easily.
00:12:30.720
Ultimately, we need to consider splitting our data models and responsibilities, maintaining separation to safeguard functional conditions and grow applications.
00:12:38.480
Next, let's briefly discuss DCI. DCI means Data, Context, and Interaction; central to this is the idea of separating behavior.
00:12:46.000
Data objects hold only data, and it's critical to make clear distinctions among various roles.
00:12:54.720
You could have a landlord and tenant who both share attributes but differ based on context.
00:13:01.680
Your roles extend via initializing the context to encompass defining responsibilities.
00:13:08.720
Upon executing the context, roles and responsibilities become apparent and are contextually managed.
00:13:17.040
A foundational example might be to rent a place to a tenant secured under the landlord’s role.
00:13:26.000
The layer of behavior should then enhance the clarity of how the system operates.
00:13:36.320
The last point to discuss, which might be slightly overboard, is treating web apps as sole objects while hiding persistence.
00:13:44.560
There are remarkable examples such as 'Candy' and 'Ambition,' which encapsulate persistent capabilities in a seamless manner.
00:13:53.520
Candy demonstrates a way to use transparent persistence in MongoDB. You can create a conference class and include it.
00:14:02.560
It automatically connects to localhost and manages everything involved in creating objects, with only a few lines of code.
00:14:10.880
Another interesting approach involves Ambition, which allows Ruby-like syntax for persistent queries without revealing underlying technologies.
00:14:18.160
You can write readable queries, enabling you to translate them seamlessly to SQL or other structures.
00:14:25.040
Both approaches facilitate efficient programming experiences and enhance flexibility.
00:14:33.440
They challenge conventional methods of persistence and encourage fresh perspectives.
00:14:42.720
I encourage you to play around and discover innovative ways to navigate persistence challenges.
00:14:51.440
Consider whether to decouple models from persistence and explore alternatives.
00:14:58.720
Learn from the features of various persistence layers.
00:15:05.600
PostgreSQL, for example, supports array columns useful for tagging.
00:15:12.160
Reflect on the concepts of creators, composition, DCI, or any ideas that arise.
00:15:18.720
Please check out 'Personal Names Around the World.' That's the second funny picture I keep using, representing us engineers.
00:15:26.560
Let's play with these concepts, experiment, and have fun!
00:15:48.960
The questions will stand between you and lunch, so I will take the blame, but if you have any, please go ahead!
00:15:54.799
If not, you can find me afterwards. Skip your next two coffees and buy a five-dollar book from Avdi Grimm, which is absolutely awesome!
00:16:03.158
Watch Uncle Bob's talk; you might disagree with it, but do watch it. There’s a great article by Piotr Sonita about making Active Record models thinner.
00:16:11.440
Check out the presentation from Martin Fowler, a great Ruby Rocks podcast, and there’s a hackathon next weekend in Warsaw that I organize.
00:16:19.440
If you want to come and help a non-profit organization to create amazing projects, we had one in December that was fantastic.
00:16:26.880
I'll be around; I'm wearing red like Santa Claus to hide the bloodstains.
00:16:34.560
Please find me, and enjoy the party!
Explore all talks recorded at wroc_love.rb 2012
+7