Talks
Events. Events. Events!
Summarized using AI
Events. Events. Events!
by Anton DavydovThe video titled "Events. Events. Events!" presented by Anton Davydov at RubyConf TH 2019 discusses the concept of event sourcing and its implications for software architecture. The presentation highlights the shift from focusing on current state data in applications to the significance of capturing events to understand the application’s history and state at any given time.
Key Points Discussed:
- Introduction of Event Sourcing: Event sourcing revolves around the idea of logging events that happen in an application rather than merely tracking the current state. This allows for historical context and the ability to reconstruct states from events.
- Types of Events: Examples provided include commits in version control systems and user actions in e-commerce applications, both of which demonstrate the importance of tracking changes over time.
- Storing Events: Various storage options for events are discussed, including Redis, Kafka, and SQL databases. The nature of events must be immutable, meaning once an event occurs, it cannot be changed, but new events can be created to reflect updates.
- Data Evolution: Event schemas need to evolve over time while maintaining historical integrity, allowing for backward-compatible changes.
- Projections: The concept of projections is introduced, whereby the current state can be derived from stored events through a predictable processing function. It addresses concerns over performance regarding the accumulation of events over time.
- Real-world Applications: Examples of event sourcing applications include financial transactions, version control systems, and audit trails. These help illustrate how detailed event logs can enhance debugging and state reconstruction.
- Challenges: The complexities of implementing event sourcing include debugging difficulties, version compatibility, and regulatory compliance related to data privacy.
- Advanced Techniques: Methods for real-time data presentation, like synchronizing databases and using WebSockets for immediate updates, are discussed.
- Implementation in Ruby: Libraries such as Rails Event Store and Sequent are recommended for implementing event sourcing in Ruby applications.
In conclusion, Anton emphasizes that while event sourcing can be complex, it brings significant advantages in managing data and understanding application functionality over time. Practicing these techniques, along with fostering a deeper grasp of business events, can make event sourcing a vital architectural strategy in software development.
00:00:07.170
Hello everyone! Hello Thailand! I want to thank you for having me here.
00:00:15.640
It's my second time in Thailand and my first time in Bangkok. Yesterday, we tried to install Party Jam.
00:00:23.260
I found it, but I'm not sure if I really want to install it. So, this is not my first trip to Asia.
00:00:31.960
This is why I really know what jet lag is. It’s tough because I have no idea whether to wake up at 4 a.m. or go to sleep at 3 p.m.
00:00:44.680
Yesterday, I was inspired by a team I really love. I tried to find something interesting on the Internet.
00:00:55.930
Unfortunately, I found nothing useful. I thought perhaps looking for 'Quest of Rubies' would be super complicated on Google, and I tried to find Ruby Nia but wasn't sure if it would be helpful.
00:01:09.119
So, okay, my name is Anton. I am from Moscow. You can find my contact information if needed.
00:01:15.130
I'm a top talent architect, and I can use this opportunity to share some insights. It's my favorite topic involving architecture.
00:01:32.920
Some people know me as the guy who works on Crimea and strong systems. I really love open source. Right now, I’m a core developer.
00:01:58.500
However, I know the theater world as well. Sometimes, I stream on Twitch. You might want to know that I really love stickers.
00:02:07.350
I have a collection of stickers, and I also love burritos. I even have a photo of myself in Minsk enjoying a burrito. I love that photo because I look like a person who could be on a Japanese TV show.
00:02:28.900
If you want to talk with me, we can discuss coffee, beer, or psychology. I have a Nintendo Switch, and I really love to draw images for our presentations.
00:02:49.480
The first drawing was done by me, and I also enjoy writing good stories.
00:03:04.090
I expected to be like a certain type of storyteller, but unfortunately, reality turned out to be a bit different.
00:03:18.820
However, I consider myself lucky because I got help from my friend Georgie. He is really quiet and sometimes asks insightful questions.
00:03:37.900
Today, the title of my talk is 'Events.' We will discuss these in general. In the real world, we encounter numerous events.
00:03:56.620
As developers, we often need to capture these events and take action. For example, does anyone know what a commit is?
00:04:11.230
Indeed, we can say that each commit represents something that happened. For instance, I have a dynamic contributions application, and we keep a list of commits.
00:04:28.810
Each commit contains a description of changes, metadata, and information about what happened.
00:04:43.540
If we open a commit from a repository, we see that it includes a description of changes, the associated files, and metadata about who made changes and when.
00:04:57.310
Another example of events is what happens when a user tries to purchase an item in an online shop application.
00:05:11.790
We need to track exactly what the user added to their order and what was deleted from it.
00:05:26.200
We have to sometimes work with deleted activity. My favorite topic is tracking deleted data in the database.
00:05:44.340
Unfortunately, in our applications, we mostly deal with the current state, meaning we focus only on what is happening at this moment.
00:05:57.310
In this case, we say our events reflect our actions, and the current state is our data in the database.
00:06:08.190
This leads us to the point that we often look at current data rather than considering how to store event histories.
00:06:24.880
So, you might be asking, what would happen if we changed our approach to start storing events instead of focusing only on the current data?
00:06:45.540
In this case, we would enter the realm of event sourcing. Today’s discussion covers both event sourcing and its significance.
00:07:04.080
The first rule of event sourcing is that nobody knows what event sourcing is. The second rule is that you must store what happened with the application.
00:07:32.110
This involves capturing events rather than simply storing the current state.
00:07:49.300
The first question you should ask yourself is where to store these events. You can keep a list of events or a chain of events.
00:08:12.770
In event sourcing, event storage can take various forms—it can be Redis, Kafka, MongoDB, or SQL databases.
00:08:33.720
Event storage has certain rules: the first is that an event must have occurred in the past.
00:08:52.850
We may not know how long ago it took place, but it’s important to remember that it happened.
00:09:07.400
The next rule is that events should be immutable. We cannot update, modify, or delete events once they occur.
00:09:20.370
Instead, we can only create new events to adjust for changes or updates.
00:09:41.740
Let’s consider a simple example of event storage: when a user is created, we log that event. Later, if we want to update the user's information, we don’t change the initial event but create a new one.
00:10:05.800
The next challenge in event storage is ensuring that we have valid data in each event.
00:10:21.570
This means validating data against our database before we store an event.
00:10:41.040
The second rule of event sourcing is data evolution. This implies that events must adapt over time without altering the history.
00:10:58.250
Documentation must remain unchanged while allowing increased complexity in the application.
00:11:15.110
Data evolution requires us to accommodate different event schemas concurrently.
00:11:33.070
It's crucial to remember that the application could experience significant changes over time.
00:11:53.290
Now, let’s move on to how we can derive the current state from an event store.
00:12:12.580
In event sourcing, we use projections to calculate specific states from the event history.
00:12:29.240
Projections act as functions that take the list of events and an initial state, then compute a specific state.
00:12:43.180
In this context, the projection function is inherently a pure function, and its behavior is predictable.
00:13:03.100
To illustrate, after a user is created, we can accumulate events and check if the user has been updated.
00:13:22.670
The concern, however, is that recalculating the current state every time can slow down performance.
00:13:37.320
Especially when events can span 2 or 10 years, handling accumulation can become a hefty task.
00:13:56.950
In Ruby or other languages, performance issues may arise, but there are ways to resolve this.
00:14:09.960
We could introduce abstraction with colored streams to better manage smaller sets of events.
00:14:26.020
For instance, with two different event sources, we could create specific streams for individual sources.
00:14:45.840
This would allow us to only process relevant events associated with each aggregate.
00:14:59.470
Additionally, we could implement snapshots of the current data state to improve retrieval speed.
00:15:14.800
These snapshots could be taken at intervals, allowing us to quickly access recent states from a database.
00:15:34.360
The general advantage here is that it enables us to leverage different kinds of databases for caching as well.
00:15:45.910
For instance, relational databases serve effectively for searching and indexing large datasets.
00:16:04.340
Conversely, document-oriented databases might be more useful for presenting detailed content.
00:16:20.190
Now let’s delve into real-world examples of event sourcing, particularly in e-commerce systems.
00:16:38.450
During order and checkout flows, it's crucial to track what actions were taken, making it easy to debug and recalculate state.
00:16:59.200
Event sourcing proves valuable in domains dealing with money, as it allows for detailed auditing of transactions.
00:17:15.580
It is also applicable in version control systems, like Google Docs, where changes can be rolled back.
00:17:31.920
Additionally, tracking systems often utilize event sourcing to calculate statistics about occurrences.
00:17:47.800
Regarding frameworks like Redux, it's important to note that event sourcing and event-driven architectures differ.
00:18:03.170
In event sourcing, we store events that occurred in the system, while in event-driven architecture, events happen and impact all components.
00:18:20.160
A good example of event sourcing can be seen in blockchains, which serve as excellent references for how this architecture functions.
00:18:36.800
Now, let’s explore the pros and cons of event sourcing. First, communicating with domain experts is easier.
00:18:54.650
We can clearly understand business events since detailed logs can be generated automatically.
00:19:12.640
Event sourcing allows us to recover lost system states whenever needed, but this comes at a cost.
00:19:30.170
It also shifts developers' mindsets; we no longer think in terms of table-based storage but rather of changes in the system.
00:19:47.890
However, event sourcing has drawbacks; it can be complicated to implement and is not popular among some developers.
00:20:01.380
Additionally, debugging can be challenging since tracing through numerous events may become cumbersome.
00:20:17.220
Version compatibility remains a concern, particularly if migrations require previous versions to be maintained simultaneously.
00:20:33.030
In Europe, data privacy regulations also impose restrictions on event management and deletion.
00:20:49.800
Now, let’s address some advanced topics in event sourcing.
00:21:07.400
When we want to display data as quickly as possible, we can apply two techniques for real-time data presentation.
00:21:29.970
The first involves synchronizing the client database with the event database as events occur.
00:21:47.490
The second strategy involves using WebSockets or long-polling to keep clients updated on changes in real time.
00:22:07.620
Another common question is how to ensure relevance ordering in a multi-application environment.
00:22:26.840
We may need to implement distributed locks or other strategies to solve issues of event collisions.
00:22:45.290
It’s also essential to think about how to deactivate or delete related events, particularly concerning data privacy.
00:23:05.800
Thus, encryption plays a role in preserving sensitive information while allowing for valid deletions.
00:23:20.100
If we have event lists, lineage is essential in efficiently retrieving information, like specific user activities.
00:23:38.440
We can achieve this by using techniques such as indexed queries and read models.
00:23:56.060
Finally, let’s talk about implementing event sourcing in Ruby. The most popular libraries available include Rails Event Store.
00:24:12.680
Also, Sequent offers a comprehensive list of features related to event sourcing.
00:24:25.900
Additionally, you can build your components if you desire a custom experience, leveraging libraries such as Kafka.
00:24:39.760
In regulated industries, incorporating event sourcing guarantees better control over changes, enabling audits and validations.
00:25:02.040
As I share this knowledge, using concepts from workshops and stickers has been instrumental in conveying intricate details.
00:25:24.540
I also created an experimental repository demonstrating how to implement these techniques effectively.
00:25:45.820
In conclusion, while event sourcing can be complex, it provides significant advantages for long-term data management.
00:26:07.100
The key is to understand the business events and leverage these techniques to handle complexity in distributed systems.
00:26:28.560
Ultimately, though complicated, mastering event sourcing can lead to better data management and clearer insights into application histories.
00:26:46.550
I believe event sourcing will become even more relevant in the future, and I encourage you to explore this area further.
00:27:06.460
Thank you all for your attention, and I'm happy to take any questions!
Explore all talks recorded at RubyConf TH 2019
+3