Scalable Deployments - How we Deploy Rails app to 100+ Hosts in a Minute

by Shota Fukumori

In his talk at RubyConf 2014, Shota Fukumori discusses scalable deployments, focusing on how Cookpad, Japan's largest recipe-sharing website, improved its deployment process to enhance efficiency. The video emphasizes the challenges faced with existing deployment tools like Capistrano, particularly as the infrastructure grew to over 120 app servers. Key points include:

Frequent Deployments: Cookpad deploys its application multiple times daily, especially during peak times, requiring developers to deploy independently.
Existing Bottlenecks: Utilizing Capistrano with SSH led to increasing delays, with deployment times extending past ten minutes, frustrating developers.
Introduction of Mamiya: A new tool named Mamiya was developed to address these delays, allowing deployment across 140 servers in less than a minute.
Deployment Process: Mamiya utilizes a structured approach with master nodes and agents, managing packaging and environment preparation to facilitate faster rollouts.
Rollback Capability: The system includes quick rollback options in case of issues post-deployment, enhancing confidence in releasing updates.
Future Improvements: Fukumori highlights continuous development and refinement of Mamiya to meet the evolving needs of deployment processes, ensuring scalability as Cookpad grows.

In summary, Mamiya not only streamlines the deployment flow but also allows developers at Cookpad to focus more on their coding tasks rather than getting bogged down in deployment logistics. This development is a significant step towards addressing the challenge of scaling deployments effectively in a rapidly growing web environment.

00:00:15.970 Thank you for having me today. I will talk about scalable deployments and how we deploy our web application at Cookpad.

00:00:28.050 Today, I will focus solely on deployment, so let's dive into that. My name is Shota Fukumori, and I'm from Japan.

00:00:39.220 I'm currently one of the youngest Ruby coaches at the age of 17. I've worked hard to gain my parents' approval to be here today, and I want to give them peace of mind about my journey in technology.

00:00:59.980 At Cookpad, we're running the largest recipe-sharing website in Japan, and we utilize a significant infrastructure that includes over 120 app servers. Deployments for us need to be efficient and effective.

00:01:15.100 We perform deployments of our application many times a day, especially during peak usage times. Our developers need the ability to deploy independently, ideally during working hours.

00:01:25.990 Next, let me outline our deployment flow. We utilize a continuous integration approach where we perform build checks. If the build is successful, it will automatically deploy to a staging server before going live.

00:01:38.800 In the past, we've used Capistrano combined with SSH to manage our deployments. However, this setup began to create a bottleneck as our server count increased, leading to slower deployments.

00:01:55.150 We discovered that our deployments were taking longer than expected. Each deployment could stretch to over 10 minutes, causing frustration among developers who wanted to quickly release changes.

00:02:09.240 At one point, the deployment process became a hindrance rather than a help, with developers often waiting for teammates which could result in additional delays.

00:02:18.480 Therefore, we needed a faster, more efficient way to handle our deployment process. We started looking for ways to enhance our current toolset.

00:02:30.950 This led us to explore how we could build a new deployment tool, which we named Mamiya. With Mamiya, we could greatly accelerate our deployment process.

00:02:43.800 Mamiya allows us to achieve deployment times of less than a minute across all our 140 servers, significantly reducing the time developers spend waiting on deploys.

00:03:02.100 In exploring Mamiya, we found that using it instead of traditional Capistrano deployments streamlined the entire process, resulting in quicker and more reliable rollouts.

00:03:13.659 This tool was developed with considerations for scalability and speed, allowing our team to focus more on coding rather than managing deployment bottlenecks.

00:03:27.460 Let me walk you through how Mamiya operates. Essentially, it handles packages that need to be deployed, ensuring that each package goes through stages of preparation before it reaches production.

00:03:46.800 Mamiya is configured to handle multiple servers efficiently, and during deployment, it utilizes a structure of master nodes and agents for better distribution of tasks.

00:04:02.280 The deployment flow involves several steps, including packaging, preparing the deployment environment, and finally switching over to the new code seamlessly.

00:04:15.600 We also ensure that the entire process allows for quick rollbacks if any issues are detected post-deployment. This greatly enhances our confidence when rolling out new features or updates.

00:04:32.350 Through implementing these changes, our deployment times have improved drastically, allowing for precise and timely updates without impacting user experience.

00:04:50.570 In summary, while Mamiya is still evolving, it addresses many of the persistent problems we faced with our earlier deployment methods.

00:05:03.220 The focus now is on continuing to improve this tool further, understanding that the needs of our deployment processes will only grow into the future.

00:05:10.760 We hope to keep enhancing Mamiya, continuously adapting it to ensure our deployments remain efficient, safe, and scalable as our service grows.

00:05:22.230 Thank you for listening, and I'm excited to see how we can further optimize our deployment processes in the near future.