00:00:21.720
Today, we're going to be talking about service architectures for mere mortals. I use the term 'mere mortals' and not something like 'service architectures for dummies' because nobody here, whether in this room or watching the video later, is a dummy for not knowing this. This stuff is very complex, and we're all trying to get through it. We're all trying to do a complicated job where the answer to every question is 'it depends.' So, it's okay not to know a lot of these things. Just recognize that we are all mere mortals.
00:00:38.930
Some of the topics we're going to cover today are: what service-oriented architecture is, a comparison with our traditional Rails monolithic app, some communication patterns amongst these decentralized services, and the trade-offs of choosing one pattern over another.
00:01:05.940
To start off, what is service-oriented architecture? The problem is, there is no single definition. Now, I say that's a problem, but there are good and bad aspects to this. Let's start with some of the downsides. One is that it's hard to Google. It's not that you won't find any information about it; the problem is you're going to find way too much information. You'll be overwhelmed by the amount of information available on service-oriented architecture, but a lot of it may not be actionable for you as a Rails developer. Additionally, it can be hard to figure out what applies to your specific circumstances.
00:01:38.429
Sometimes, you'll come across information that is too specific or focused on a different ecosystem. Service-oriented architecture isn’t that common in the Rails ecosystem compared to environments like Java or .NET, so you might find resources that are too focused for someone in the Rails ecosystem. When you encounter overly specific material, you might end up diving down a Wikipedia rabbit hole, trying to map concepts to those that exist within Ruby gems.
00:02:00.880
Other times, you may stumble upon content that is too vague or theoretical, which proves to be high-level with very little actionable information. Many authors avoid being too specific to avoid gatekeeping, but that leads to a lot of other confusing content that states if you’re not implementing services in a certain way, then you’re not really doing it. This unfortunate gatekeeping leads to unnecessary complexity.
00:02:25.050
Moreover, there's no real convention around service-oriented architecture. As Ruby developers, we cherish our conventions; we like to install a gem and get a lot of functionality without much fuss. However, there’s no simple command like 'gem install SOA' to get a service-oriented architecture for free. After I wrote that, turns out there has been some half-baked attempt at this in Searls, but it's not real.
00:03:10.250
So, there is no solid convention around this, but there are many fun attempts, including talks like this one, and there have been a few at other Ruby-focused conferences about services, but nothing comprehensive to get you going. The bright side is that you're not locked into any particular implementation of service-oriented architecture. That’s one of the nice things about lacking conventions; you have a lot of freedom. Any components you incorporate into your system won’t lock you into a specific implementation.
00:03:44.250
However, there are common patterns you will see in service-oriented architectures. The key takeaway is that your Rails application is no longer the center of the universe. It is now just a single component of a larger distributed system.
00:04:07.430
One pattern might involve your services serving up JSON or another machine-readable format. The consumers of these services could be your end-users interacting with your API or it might be your own services consuming data from other services within your system. Sometimes, services might serve HTML rendered from various parts of different services, which are all bundled together to serve to your end users.
00:04:32.580
Some of your services could be running on Rails, while others might not be. For smaller microservices, you could be utilizing something like Sinatra or Rhoda. Additionally, when communicating with a remote service, you don’t need to use the same programming language on both sides of that conversation. For a service that requires high performance, you could use languages known for having efficient runtimes, such as Go or Elixir. In fact, they don't even have to communicate using HTTP.
00:05:00.060
Some services could be stateless, indicating they don’t keep a database. When interacting with a stateless service, you generally ask it to perform an action, such as indicating something has happened that it should respond to. The flip side of statelessness is statefulness—where data is stored somewhere, either in a traditional SQL database, in S3, Redis, or another back-end store. The specifics of how data is stored shouldn't concern you as a consumer of that service; what's important is knowing you can fetch the data.
00:05:30.000
If you are uncertain about which database to utilize while building a service, my advice is this: if you are unsure, just use Postgres. You’ll be fine. Your services can also call out to other services, meaning you don’t need to be the terminal endpoint of a dependency chain. If someone requests information from your service and you need data not present in your local store, you can reach out to other services. This is a common pattern.
00:06:00.540
This serves as a high-level overview of service-oriented architecture. In the next section, we will delve deeper into this topic and compare it with a monolithic Rails architecture.
00:06:30.000
While discussing monolithic architecture, I wanted to capture a definition from the dictionary, which, as is traditional, defines a word in terms of itself. A monolithic app is defined as being related to or resembling a monolith, where 'huge' or 'massive' are additional descriptors. This definition is interesting, as we find that monolithic Rails apps can grow quite large, often incorporating a lot of functionality over time, particularly over five to ten years.
00:07:05.000
The essence of a monolithic app is that you perceive it as a singular entity. You are developing, testing, and deploying it as a single unit. However, regardless of what a monolithic app considers itself to be, it often views itself as the most important component in the universe. Many web applications are built with that idea encoded into their DNA—whether explicitly or implicitly.
00:07:30.000
One fun aspect about monolithic apps is that they are not truly monolithic. For instance, let’s consider a user visiting your Rails app. When a user makes a request through their browser, Rails sends back a response—but we overlook many underlying processes that occur between that request and the response.
00:08:00.000
When a request comes in, the Rails app needs to determine what to send back in its HTTP response. Since the Rails app does not keep everything in memory or within its local file system, it must reach out to other components, typically a database. Thus, the database returns query results to the Rails app, which then converts the tabular data from Postgres into a format that the domain objects can handle, packaging it into HTML.
00:08:30.000
It might seem trivial to consider the database separate from the Rails app, as we typically think of it as part of the app. However, your app could also send emails, so your users receive notifications or receipts. In this case, we don’t handle SMTP directly; we send it off to a third-party email service.
00:09:00.000
We don't want to send emails during a web request because doing so adds latency. Therefore, we send the email request to a background job processor like Sidekiq. In this scenario, the Rails app communicates with Sidekiq through Redis, which acts as the intermediary.
00:09:30.000
Similarly, we need a payment processor for any production system that requires it, alongside systems for logging, error tracking, and consumer-side analytics. We might also perform caching on certain resources to enhance performance under load or boost throughput, so we may have a cache store to assist with that.
00:10:00.000
Furthermore, we probably want to implement faceted and full-text search, necessitating the use of a dedicated data store. Thus, our once 'monolithic' application, conceived as a single piece, actually comprises many individual components. Most Rails applications I have worked with in production have this structured configuration hidden behind the facade of a monolith.
00:10:30.000
To summarize, truly monolithic architecture is more about centralization. The Rails app serves as the focal point for all interactions, with most data passing through the Rails server at some point, whether originating from data stores or routing through processes like email.
00:11:00.000
In contrast, service-oriented architectures are largely decentralized. However, it's crucial to note that there are certain components that nearly every service will require to interact with. In most Rails apps, there exist what are referred to as God objects, usually pertaining to core entities, such as the user model or other vital elements regarding your business, like products or orders in an e-commerce app.
00:11:30.000
In a service-oriented approach, these God objects evolve into their own standalone services. For instance, every distributed system I've worked with has had a user service, which may include authentication or simply identification by user ID. This requirement stems from maintaining a centralized repository of user data, encapsulated in the concept of a 'single source of truth.'
00:12:00.000
Another central component could be a service registry. Many implementations exist, the simplest being a hosted hash-like structure where you register all services by name, with values being URLs or configuration information that might be necessary for communicating with the respective service.
00:12:30.000
When utilizing a service registry, each deployed service needs only configure the registry URL for communication. This makes the system effectively decentralized.
00:13:00.000
For example, if we consider the e-commerce application architecture, each service may have its own processes for handling web requests, processing background jobs, etc. Each service has its own data stores, meaning they can manage databases and caches without interference from one another.
00:13:30.000
You should undoubtedly avoid sharing a database among multiple services unless absolutely necessary, as this can lead to numerous complications. One issue arises when multiple services require shared migrations. When deploying the updates, services might need to reboot, and you can quickly run into connection limits if you're using a database with a capped connection count.
00:14:00.000
Each service typically has different needs regarding its data. For instance, if a developer in the catalog service determines that calculating the average customer review on every request is too resource-intensive, they may choose to denormalize that data and add a column to the products table for quick access. However, if another service, like the cart service, starts using that column, and later it gets removed, it may cause failure in the cart service.
00:14:30.000
This highlights the importance of ensuring that each service maintains its own database in order to minimize the impact of such issues.
00:15:00.000
As we progress into the communication patterns between services, the most prevalent pattern I've observed involves using a machine-readable format, typically JSON, sent over HTTP. In contrast, if you're interacting with a Java Enterprise Service, you might be utilizing XML, which is still common in high-scale situations. However, when dealing with high-performance scenarios, JSON might become too heavy.
00:15:30.000
In such cases, you might opt for lighter formats, such as MessagePack or Google's Protocol Buffers. Imagine a user-facing webpage that builds an HTML page using data from the user's catalog and cart services. This interaction leverages user services to identify the user utilizing cookie data, retrieves the product catalog from the catalog service, and checks what items are in the user's cart from the cart service.
00:16:00.000
Notice how we name these services: 'user service,' 'catalog service,' and 'cart service.' These names maintain specificity while avoiding lower-level references to database tables. Avoiding discussions of services solely as database tables is vital; if we do that, we end up merely dispersing a database across services without achieving true service-oriented architecture. Although there may be overlapping data requirements between the catalog and cart services, each will still need a distinct view.
00:16:30.000
For instance, when a product is added to a cart, the cart service may reach out to the catalog service. However, HTTP doesn’t work for every situation since it’s a point-to-point protocol. If one service performs an action that other services need to know about, there is uncertainty regarding whose responsibility it is to share that information.
00:17:00.000
One additional challenge of using HTTP for everything is performance. You either need to maintain a connection pool for every service you communicate with, or you incur the costs of establishing a new connection, which involves TCP handshakes and TLS negotiations. While this may not sound significant, when thousands of requests are made every second, it can add significant overhead.
00:17:30.000
Furthermore, HTTP requests typically involve waiting for a response. In languages like Ruby, where libraries are optimized for synchronous I/O, ignoring the response isn't feasible. When you send an HTTP request, it must return a response, which can extend your overall processing time. Conceptually, when a service needs to communicate with multiple services, it appears to send all requests simultaneously.
00:18:00.000
But in reality, it waits for each response sequentially. This can lead to significant delays, potentially ranging from milliseconds to several seconds.
00:18:30.000
In Ruby, asynchronous I/O is complex. If you want to send multiple requests and await their responses, it's challenging and often requires specific gems tailored for such asynchronous tasks. Additionally, you must navigate dependency management for your calls.
00:19:00.000
The common resolution to eliminate these synchronous dependencies is employing a message bus. Instead of having Service A communicate directly with Services B, C, and D, it pushes a message to a message bus, which Services B, C, and D can also consume. This decouples them from the dependency on each other.
00:19:30.000
This pattern is known as publish/subscribe or 'pub/sub.' Picture it similar to a Slack workspace that is entirely bot-operated. When one service sends a message to the message bus, all subscribed services receive that notification. When a service receives such a message, it can take action, like updating internal data or sending responses back into the message queue.
00:20:00.000
The bonus of this system is that your service doesn't require being reachable via the Internet; it does not need to process incoming connections. Rather, it consumes messages from a known source, which means it can operate wherever—whether on a cloud platform or a Raspberry Pi on your desk. This structure eliminates concerns like domain names or static IP addresses.
00:20:30.000
This design can also help mitigate the age-old developer complaint of 'it works on my machine.' If someone reports a bug on their machine, you can reproduce the environment, linked to production, and consume messages as they would.
00:20:55.000
As we wrap up communication patterns, we observed the prevalence of machine-readable formats, such as JSON or HTML fragments sent over HTTP, and asynchronous communication via message buses. While not exhaustive, this encapsulates about 95% of the use cases I've encountered.
00:21:30.000
Next, let's dive into the trade-offs of adopting a service-oriented architecture over a monolithic Rails app. A significant trade-off is latency. Latency refers to the time taken to perform an action once, while throughput is how often you can perform that action within a specified timeframe.
00:22:00.000
Distributing your work across multiple services can yield increased throughput, yet the more services you need to interact with for a single action to complete, the longer your overall delay. In essence, you're shifting work from your local machine to another machine, which means accounting for time spent serializing and transmitting data over the network.
00:22:30.000
Typically, this latency is measured in milliseconds—affordably negligible at first. However, as your system grows and you add multiple layers of services, this latency can compound significantly.
00:23:00.000
For example, in a service graph at Netflix, the edge services on the far left communicate with a myriad of internal services. Each service's dependency accumulates, leading to compounded latency based on the collective downtime of these services.
00:23:30.000
Another essential aspect to navigate when transitioning to microservices is team structure. By organizing around these services, there might be minimal overlap in functionality or responsibility across teams. These silos can hinder communication, requiring mitigation strategies to maintain cooperative dialogue among teams.
00:24:00.000
Distributed systems often pose significant maintenance demands. Once you embark on this journey, you may need dedicated personnel to ensure that all services remain connected, working with developers and managers to maintain consistent interfaces between components.
00:24:30.000
One of the largest challenges with distributed systems is the loss of atomicity guarantees. In traditional databases, we utilize transactions to ensure that either all operations succeed or none do. However, this condition is difficult to guarantee when transactions span multiple systems.
00:25:00.000
For instance, consider when a user adds items to their shopping cart. A message gets published stating that user A is adding items. If the inventory service determines that there is insufficient stock, it can't fulfill the request, leading to compensating actions where the cart service must handle errors and notify the user accordingly.
00:25:30.000
This example illustrates the need to anticipate rollback mechanisms and ensure every action published results in clear responses from the dependent services. Just like you must write up and down methods for ActiveRecord migrations, equivalent strategies are necessary for external interactions.
00:26:00.000
Uptime also becomes a crucial trade-off. Every individual service’s uptime directly impacts the overall system's uptime. If several services depend on others, the total downtime experienced increases sharply. If you have four services, and each service has three nines of uptime, the cumulative downtime rapidly compiles.
00:26:30.000
For example, with three services at 99% uptime, your overall service could incur several hours of downtime per year. Layering additional dependencies exacerbates this issue. If one service fails, it creates a cascading failure affecting all dependent services.
00:27:00.000
It's essential to understand the implications of moving towards a decentralized architecture, so you can allocate resources for mitigation strategies surrounding downtime. A silver lining emerges in terms of deployment speeds: while a monolithic Rails app might take minutes to hours to deploy, individual services can often deploy in mere seconds.
00:27:30.000
For example, one of our smaller services is able to build and deploy in under 30 seconds. This quick turnaround can prove crucial, particularly for fixing critical bugs after deployment.
00:28:00.000
Testing and developing a service can present unique challenges. Stub out dependencies, similar to instances where the VCR gem is utilized, where stubbing is necessary when integrating third-party services, now must extend to your own services as well. This may require setting up a separate development environment or cluster when conducting CI against multiple services.
00:28:30.000
However, when employing asynchronous communication through a message bus, testing generally becomes less cumbersome. You only need to verify that responses to messages are received as expected, enabling you to focus on individual service outcomes.
00:29:00.000
For example, while working on a 'cart service' that handles an 'add to cart' message, you might ensure that the respective product is correctly added to the user's cart—making the assertion process more direct.
00:29:30.000
In conclusion, we have examined numerous trade-offs associated with transitioning from a centralized architecture to a decentralized one. I invite anyone interested to discuss further, as I truly enjoy engaging with fellow developers on these topics.