Talks
Speakers
Events
Topics
Sign in
Home
Talks
Speakers
Events
Topics
Leaderboard
Use
Analytics
Sign in
Suggest modification to this talk
Title
Description
Last year at this very conference John Duff spoke about how Shopify scales while maintaing one of the longest lived and largest Rails deployments, and how we affront the challenges that come with growth. Shopify in 2013 became more than twice the size in every single aspect; requests per minute, GMV, merchants, number of developers, etc. After CyberMonday 2012 it became clear that if we wanted to survive CyberMonday 2013 we needed to spread the load across more than a single huge database, and move to a model of smaller databases to enable horizontal scalability. This is the story of how, in 2013, we more than doubled the number number of databases that power Shopify, and all the challenges that come along when sharding a living, breathing and money producing Ruby application. Help us caption & translate this video! http://amara.org/v/FG3p/
Date
Summarized using AI?
If this talk's summary was generated by AI, please check this box. A "Summarized using AI" badge will be displayed in the summary tab to indicate that the summary was generated using AI.
Show "Summarized using AI" badge on summary page
Summary
Markdown supported
In this video, Camilo Lopez from Shopify presents a detailed account of how the company successfully implemented sharding in their Rails architecture in 2013. The need for this transition arose from exponential growth in traffic and customer count, particularly after the heavy load experienced during CyberMonday 2012. Shopify had to evolve from a single large database to a scalable model with multiple smaller databases. ### Key Points Discussed: - **Company Background**: Shopify is an e-commerce platform serving approximately 100,000 active shops, with high-performance infrastructure based on Ruby on Rails and MySQL. - **Scaling Challenges**: The platform handled 150,000 requests per minute on average, with peaks up to 400,000, which necessitated a shift to sharding to improve performance and reliability. - **Sharding Complexities**: Introducing multiple databases complicated the Rails application which was traditionally designed for single-database operations, leading to challenges in data grouping and transactional integrity. - **Initial Strategy**: Before sharding, Shopify focused on optimizing MySQL and tuning their hardware until expansion needs prompted the shift to a sharded architecture. - **Implementation Process**: A dedicated team was assigned to focus on sharding, which included substantial preparation to prevent complexities during migration and maintain data consistency. - **Data Migration and Model Changes**: The migration involved redefining the data structures to include shop IDs and implement sharded model connections. A structured team approach ensured both stability and ongoing development. - **Performance Optimization**: Real-time performance was crucial, especially during peak times, thus advanced concurrency management and transaction handling were integrated. - **Success Metrics**: By Cyber Monday 2013, Shopify successfully handled up to 100,000 RPM with excellent response times, proving that their sharding approach effectively supported their significant increase in user demand. ### Conclusions: Lopez concludes the presentation by emphasizing the importance of thorough planning and structural awareness in application architecture which laid the groundwork for Shopify’s ongoing adaptability. They also highlight the profound benefits of the sharding strategy, ensuring robust performance and service reliability even amidst fluctuating traffic demands.
Suggest modifications
Cancel
