Talks
Speakers
Events
Topics
Sign in
Home
Talks
Speakers
Events
Topics
Leaderboard
Use
Analytics
Sign in
Suggest modification to this talk
Title
Description
RailsConf 2017: Beyond Validates_Presence_of: Ensuring Eventual Consistency by Amy Unger You've added background jobs. You have calls to external services that perform actions asynchronously. Your data is no longer always in one perfect state-- it's in one of tens or hundreds of acceptable states. How can you confidently ensure that your data is valid without validations? In this talk, I’ll introduce some data consistency issues you may see in your app when you begin introducing background jobs and external services. You’ll learn some patterns for handling failure so your data never gets out of sync and we’ll talk about strategies to detect when something is wrong.
Date
Summarized using AI?
If this talk's summary was generated by AI, please check this box. A "Summarized using AI" badge will be displayed in the summary tab to indicate that the summary was generated using AI.
Show "Summarized using AI" badge on summary page
Summary
Markdown supported
In her talk titled "Beyond Validates_Presence_of: Ensuring Eventual Consistency" at RailsConf 2017, Amy Unger addresses the challenges of ensuring data validity in a distributed system, acknowledging that data can exist in multiple states due to the use of background jobs and external services. Unger's journey began with simple data management through Active Record validations, but evolving use cases in consulting and engineering led her to recognize the limitations of conventional validation methods in the face of asynchronicity. Key points from her presentation include: - **Causes of Data Inconsistency**: The introduction of background jobs can lead to scenarios where data is valid for a limited time, suggesting that reliance on validations alone is insufficient. - **Prevention Strategies**: To maintain data consistency, Unger recommends sophisticated error handling strategies, such as using retry mechanisms, implementing idempotent actions, and considering rollback approaches with caution when external systems are involved. - **Eventual Consistency**: Acknowledging that not every data point can be perfect at any moment, she emphasizes the importance of accepting that a product may exist without a billing record temporarily, with an expectation that data will ultimately converge on valid states. - **Monitoring and Detection**: Effective detection of data inconsistency requires thoughtful use of timestamps in databases and sophisticated querying to reconcile states across services. She suggests implementing a structured approach to monitor and respond to data anomalies using SQL queries effectively. - **Using Event Streams**: As a part of her technical recommendations, Unger discusses the benefits of using event streams to track application processes, simplifying the auditing of transactions and maintaining a consistent flow of information across systems. Unger concludes her talk by stressing the importance of understanding the trade-offs involved in building out complex data validation systems, noting that while more engineering effort is necessary to ensure data correctness, it is essential for critical aspects of applications such as billing and security. She encourages developers to tailor their approach based on the specific context and potential business risks associated with their applications.
Suggest modifications
Cancel
