Suggest modification to this talk

Title

Description

Tobias Lutke wrote the first line of code for Shopify nearly 10 years ago to power his own Snowboard shop. Two years later Shopify launched to the public on a single webserver using Rails 0.13.1. Today Shopify powers over 40k online stores and processes up to half a million product sales per day across the platform. Over 30 people actively work on Shopify which makes it the longest developed and likely largest Rails code base out there.
This is the story of how Shopify has evolved to handle its immense growth over the years. This is what getting big is all about: evolving to meet the needs of your customers. You don't start out with a system and infrastructure that can handle a billion dollar in GMV. You evolve to it. You evolve by adding caching layers, hardware, queuing systems and splitting your application to services.
This is the story of how we have tackled the various scaling pain points that Shopify has hit and what we have done to surpass them, what we are doing to go even further.

Help us caption & translate this video!

http://amara.org/v/FGb8/

Date

Summarized using AI?

If this talk's summary was generated by AI, please check this box. A "Summarized using AI" badge will be displayed in the summary tab to indicate that the summary was generated using AI.

Show "Summarized using AI" badge on summary page

Summary

Markdown supported

The video "How Shopify Scales Rails" presented by John Duff at Rails Conf 2013 details the evolution of Shopify, a hosted e-commerce platform built on Ruby on Rails, from its inception to its current scale. The talk highlights how Shopify has addressed significant growth challenges over the years through strategic scaling methods.

### Key Points Discussed:

- **History and Growth of Shopify**: 
  - Launched publicly in 2006, Shopify has grown tremendously, currently supporting over 40,000 online stores and processing up to half a million product sales daily. 
  - The codebase has seen continuous improvement; Shopify has upgraded from Rails 0.13.1 to Rails 3.2 without any rewrites.

- **Technical Architecture and Tools**: 
  - The architecture involves 53 app servers, 1,590 unicorn workers, and different caching mechanisms like Memcache and Redis. 
  - They use various tools, including New Relic for real-time monitoring, Splunk for log management, and StatsD for metrics gathering.

- **Scaling Strategies**:  
  - **Caching Layers**: Implemented caching strategies (like "Cachable" and "Identity Cache") to reduce database load by caching commonly requested data. 
  - **Job Processing**: Transitioned from Delayed Job to Resque for background job processing to enhance performance from 120 jobs/second to over 300 jobs/second. 
  - **MySQL Optimization**: Introduced techniques for MySQL, such as query optimization and proper index management, leading to significant improvements in performance.

- **Handling Traffic Spikes**:  
  - Discussed specific scenarios, like handling high request rates during flash sales (over 200k requests per minute), emphasizing the need for flexible scaling solutions. 
  - The importance of knowing the system characteristics and continuously measuring performance to identify issues was underlined.

- **Service-Oriented Architecture**:  
  - As Shopify grew, separating components into services became necessary to manage load effectively, such as moving image processing to a dedicated service, which allowed for tailored scaling and resource allocation.

### Conclusions and Takeaways:

- Adapting and evolving are crucial as needs change; understanding performance metrics and the system's limitations is key to successful scaling. 
- Continuous measurement and external tools play a vital role in recognizing what scaling strategies work and where improvements can be made without negative impacts. 
- The talk reinforces that scaling is not a one-size-fits-all solution; each department must be approached based on current performance demands and predictions for future growth.

Cancel