00:00:17.340
Hello everyone! I'm really happy that you made it here. I had my doubts about whether I was going to make it, but in the end, cooler heads prevailed, and here we are.
00:00:29.400
Let me just go through the formalities. My name is Karol Szuster, and I am a software engineer working at Upside.
00:00:37.680
Today, I want to talk to you about nightmare neighbors. This might sound confusing without the subtitle, but I don’t mean the guy that lives next door to you.
00:00:44.640
Although those kinds of neighbors can also cause you to lose quite a bit of sleep.
00:00:51.420
To clarify what I mean by neighbors, I am referring to the multi-tenant application pattern. Let's fill the waters here.
00:01:15.180
By a show of hands, could you tell me who here has previous experience with such an architecture? It's quite a few of you, which is great. I hope you will serve as fact-checkers, and I also hope you will learn something today.
00:01:40.860
This presentation assumes very little previous knowledge about this pattern, so I’ll begin with a small introduction.
00:01:51.960
Multi-tenancy is a software development pattern where a single instance of an application serves multiple clients. We can visualize it in a diagram.
00:02:05.700
In this case, we should probably name something different. By 'users' on the slide, I mean customers—businesses that use applications.
00:02:10.739
These are the tenants for whom we will separate data. We can contrast this pattern with the traditional single-tenant approach.
00:02:23.700
In a single-tenant model, each customer or client has a separate instance of our application.
00:02:35.340
As opposed to multi-tenancy, which aims to create the illusion for the users that they are the only ones using the application.
00:02:40.860
Quickly, I will go over why we would even want to implement multi-tenancy.
00:02:46.680
First of all, the setup is much quicker. Spinning up a new instance for each client is complicated and costly.
00:03:05.820
Maintenance becomes unscalable if we rely on the single-tenant application pattern.
00:03:11.540
Additionally, the costs of using a single-tenant application might lead to underutilization of resources.
00:03:20.579
Multi-tenancy allows us to optimize our resource usage effectively.
00:03:29.400
The pivotal concern in this pattern is how to partition the data. How do we actually isolate the tenants?
00:03:36.660
This is critical because our reputation as a company relies on maintaining this isolation without data leaks.
00:03:42.240
It's especially important if our clients are enterprises—leaks in this context are unacceptable.
00:03:49.320
There are multiple levels at which we can separate the data, and I will go through these methods one by one.
00:03:54.720
The first method is row-level partitioning, which is a common approach in relational databases. The idea is simple: each record in the database has a tenant assigned.
00:04:08.879
For implementation, you could have a tenant table in the database that links to users.
00:04:14.580
To do this effectively, I advocate for a normalized schema whenever possible.
00:04:33.360
This normalization makes it easier to work with the data without constantly having to refer to separate relationships.
00:04:55.740
As for implementing this partitioning, you can use default scopes to avoid having to repeat the same where clause in each query.
00:05:12.780
This ensures that the application injects the correct tenant information into all relevant queries.
00:05:19.680
Extracting the tenant can depend on your particular use case, each with its advantages and disadvantages.
00:05:26.880
You could extract it from the host, subdomain, or request headers.
00:05:33.000
To actually set the current tenant, there's no standard method; it generally hinges on the thread current object.
00:05:41.640
You could use request store, or something in ActiveSupport called current attributes.
00:05:49.959
Having a current tenant per request allows you to populate this for each request.
00:06:01.500
This method allows you to avoid special infrastructure and maintains minimal overhead.
00:06:13.320
Creating new tenants using this approach is straightforward. It can be done simply by adding records into the database.
00:06:27.420
However, the application holds full responsibility for separation. We must ensure that all requests are scoped properly, including validations, to maintain isolation.
00:07:04.920
To assist with this, gems like ActsAsTenant or ActiveRecord MultiTenant are available.
00:07:10.260
Despite having these tools at our disposal, we should remain vigilant about data integrity, especially when our focus is on maintaining it.
00:07:24.000
Should we forget our workloads, this could lead to undesired behaviors like leaking data.
00:07:35.940
One approach to mitigate these risks is using row-level security in the database.
00:07:42.240
Row-level security allows us to restrict which rows can be returned based on which tenant is making the request.
00:07:51.480
To enable this, we define security policies for each table that require tenant identification for record access.
00:07:58.200
Note that not all users are subject to these restrictions. Certain roles, such as super users, may bypass this security.
00:08:06.360
It's essential not to run production databases as super users.
00:08:11.280
With the database session parameter handling, we can keep projects simpler by ensuring the correct tenant is set once per request.
00:08:21.180
However, we must also reset session parameters in Rails to prevent connection pool issues.
00:08:28.560
Utilizing middleware is a common method to extract the tenant and manage connection pooling.
00:08:36.839
This approach has both benefits and potential pitfalls.
00:08:41.399
If we forget to explicitly set the tenant, we may not retrieve the expected records, which can lead to application errors.
00:08:48.840
This failsafe allows us to restrict data returns more securely than traditional where clauses.
00:09:05.760
Despite its strengths, this introduces implicit state management issues, which we must be aware of.
00:09:15.840
The connection pool mechanism must ensure that we track the correct tenant through requests.
00:09:22.680
Scaling applications requires diligence when managing connection pool operations.
00:09:29.520
When utilizing multiple replicas, it would be wise to consider external connection pooling.
00:09:36.900
For example, PgBouncer can effectively manage database connections, especially in transaction pooling scenarios.
00:09:45.300
We can define which databases to connect when performing queries and transactions.
00:09:54.900
Another method to achieve multi-tenancy is through schema-level partitioning, where each tenant has its own schema.
00:10:05.520
Each schema contains tables representing different tenants, helping further isolate data.
00:10:12.600
Again, using the search path session parameter is crucial for managing schemas effectively.
00:10:22.560
While this method offers low effort in migration processes, we face drawbacks as well.
00:10:32.520
Migrations will need to be custom-built for every tenant, which can lead to complexity and increased scope.
00:10:39.480
Creating new tenants becomes a more involved task, transferring data to the new schema.
00:10:46.680
The Heroku platform cautions against this approach due to backup challenges involved.
00:10:52.860
Shared schema management adds another layer of complexity that can affect use cases.
00:10:59.520
Eventually, we get to database-level partitioning—where each tenant is served by a completely separate database.
00:11:11.220
While it offers clear benefits, it may also require re-establishing connections frequently.
00:11:24.000
Some tools manage this by retaining previous connections when switching databases.
00:11:29.520
Although easy to implement, hidden issues can arise as the application scales.
00:11:34.560
Database connections must be handled with caution to avoid leaking data across threads.
00:11:43.680
Horizontal sharding, where we maintain multiple databases with the same schema, offers a viable solution.
00:11:50.760
This became natively supported in Rails, showcasing a modern approach to multi-tenancy.
00:11:55.680
As with so many architecture choices, weighing pros and cons is essential.
00:12:00.840
You must consider the unique demands of your business to select an appropriate approach.
00:12:07.620
Plan early to avoid complications arising from locking yourself into a decision.
00:12:13.260
I appreciate your time and hope this discussion has been valuable.
00:12:18.960
I managed to wrap it up in time, and I'll now take any questions.
00:12:24.000
So, which of these solutions would you recommend if you need to have aggregated computations from multiple tenants?
00:12:35.100
Both the schema-level and row-level approaches allow for aggregation since they enable table referencing across schemas.
00:12:41.880
If I had to choose one, the row-level approach is the most battle-tested solution for this problem.
00:12:54.360
Many successful multi-tenant architectures like Shopify and Salesforce utilize this strategy.
00:13:01.740
Thank you for your question!
00:13:07.740
I understand the concerns about row-level multi-tenancy, particularly regarding where clause handling.
00:13:18.960
If a model is scoped to a tenant, trying to access the data without setting the tenant will result in an error.
00:13:26.340
It’s essential to remain vigilant about managing these access rules.
00:13:35.040
Yes, it's a valid concern about accessing the database not just through the application but other services.
00:13:43.020
This reinforces how important it is to mind tenant scoping in various contexts.
00:13:49.440
Thank you for your insightful remarks!
00:13:57.780
Regarding your question about database performance and indexing per tenant:
00:14:04.140
Indexing can indeed improve performance, especially when using multi-column indexes.
00:14:14.100
Partitioning inside the database could also improve query speed by reducing the search scope.
00:14:20.520
It’s an important aspect that requires long-term planning.
00:14:26.940
Additionally, using read models in a multi-tenant context could enhance data separation.
00:14:35.520
Using materialized views might structure your data to improve access.
00:14:40.680
Thank you all for engaging! If there's nothing else, I’ll wrap it up here.
00:14:46.980
Thank you again. Have a great day!