Talks
Speakers
Events
Topics
Search
Sign in
Search
Home
Talks
Speakers
Events
Topics
Leaderboard
Use
Analytics
Sign in
search talks for
⏎
Suggest modification to this talk
Title
Description
Pop quiz: How do you best make one Rails app communicate with another? How do you split one big Rails app into two smaller ones? How do you switch from a Rails app to a constellation of Rails services? Event streaming provides the most robust answer. Simple but powerful Kafka streams unlock a world of capabilities with their durability and consistency, but integrating with Rails poses challenges. Join us and learn simple reading and writing to Kafka with Rails, broader distributed systems design, and the magical transactional outbox. You'll leave with the knowledge you need to make the switch.
Date
Summarized using AI?
If this talk's summary was generated by AI, please check this box. A "Summarized using AI" badge will be displayed in the summary tab to indicate that the summary was generated using AI.
Show "Summarized using AI" badge on summary page
Summary
Markdown supported
In the presentation titled **Event Streaming on Rails** at RailsConf 2022, Brad Urani, Principal Engineer at Procore, explores the concept of event streaming as a solution for scaling large Rails applications. Urani discusses how companies can transition from monolithic Rails applications to microservice architectures through effective use of event streaming technologies, primarily using Apache Kafka. The core topics addressed in the talk include: - **Communication Challenges in Large Applications**: Urani outlines the difficulties in making large Rails applications communicate, especially when trying to split a monolith into smaller services and facilitating data replication across different geographic regions. - **Introduction to Apache Kafka**: The speaker explains Kafka as a distributed event streaming platform, highlighting its capabilities, including durability, guaranteed delivery, and message ordering, which is crucial for maintaining data consistency across systems. - **Integration with Ruby on Rails**: Practical insights are provided on how to implement Kafka within Rails using the Ruby Kafka gem. Urani describes both synchronous and asynchronous message publishing methods, emphasizing the trade-offs between speed and message durability. - **Common Use Cases for Kafka**: - Service-to-service communication, where Kafka allows for decoupled services that can operate independently, improving fault tolerance. - Multicasting events for diverse processes, such as notifying various downstream services when a user signs up. - Ensuring consistency between heterogeneous data stores like PostgreSQL, Elasticsearch, and reporting databases using event sourcing and change data capture techniques. - **Transactional Outbox Pattern**: Urani discusses the dual-write problem inherent in standard database operations and proposes the transactional outbox pattern as a solution. This technique involves writing to an 'outbox' table within a single transaction to ensure that the data published to Kafka aligns with the database entries, providing a reliable mechanism to maintain consistency. - **Operational Considerations**: The necessity for operational strategies like Kafka Connect to simplify the integration of various data sinks and sources was also discussed, particularly for maintaining historical data. In conclusion, Urani illustrates that adopting event streaming with Kafka not only boosts scalability and performance of Rails applications but also enhances their reliability through strong guarantees on message delivery and ordering. By leveraging these technologies, developers can create a more resilient architecture, paving the way for future growth and adaptability within their applications.
Suggest modifications
Cancel
