Talks
Speakers
Events
Topics
Sign in
Home
Talks
Speakers
Events
Topics
Leaderboard
Use
Analytics
Sign in
Suggest modification to this talk
Title
Description
Unpredictable traffic spikes, slow requests to a third party, and time-consuming tasks like image processing shouldn’t degrade the user facing availability of an application. In this talk, you’ll learn about different approaches to maintain high availability by offloading work into the background: background jobs, message oriented middleware based on queues, and event logs like Kafka. I’ll explain their foundations and how they compare to each other, helping you to choose the right tool for the job.
Date
Summarized using AI?
If this talk's summary was generated by AI, please check this box. A "Summarized using AI" badge will be displayed in the summary tab to indicate that the summary was generated using AI.
Show "Summarized using AI" badge on summary page
Summary
Markdown supported
In this talk, Kerstin Puschke, an expert from Shopify, discusses high availability in applications, particularly in the context of handling unpredictable traffic spikes and ensuring seamless user experiences. Puschke emphasizes the importance of offloading intensive tasks, thereby avoiding system degradation during peak loads. The main approaches explored include background jobs, message-oriented middleware, and event logs like Kafka. **Key Points:** - **Definition and Importance of High Availability:** - High availability is more than uptime; it involves ensuring users can interact with the system without degradation of service during heavy traffic. - **Background Jobs:** - Background jobs are utilized to offload tasks from the user-facing part of the system, ensuring that heavy processing does not interrupt ongoing user requests. - At Shopify, background jobs handle up to tens of thousands of tasks per second, improving system responsiveness and availability. - **Advantages of Task Queues:** - Task queues act as buffers, allowing the system to queue jobs without impacting user-facing requests. - They enable horizontal scaling by allowing multiple workers to handle tasks concurrently. - **Challenges with Background Jobs:** - Managing versioning and job parameters can lead to errors if not properly synchronized. - Handling failures and maintaining consistency is crucial for maintaining high availability. - **Message-Oriented Middleware:** - This approach separates message routing and management from service logic, allowing for a more decoupled system. - Events can be published and consumed without tight coupling, improving overall availability. - It can handle consumer downtime by queuing messages, ensuring no data is lost. - **Event Logs (e.g., Kafka):** - Event logs provide a shared append-only log, allowing multiple consumers access to the same data without losing messages. - They are useful for real-time applications and provide a single source of truth, allowing for event sourcing. - **Conclusion:** - Puschke concludes that while each approach has its complexities, they are essential for maintaining high availability in modern applications. Each strategy has its advantages depending on the architectural requirements—background jobs work well in monolithic structures, message-oriented middleware is suited for microservices, and event logs are beneficial for high-throughput needs. Through her insights and case studies tailored around Shopify’s operational needs, Puschke highlights that choosing the right tool for offloading work is crucial for sustaining high availability and accommodating growth.
Suggest modifications
Cancel
