User Experience (UX)
Chanko - How Cookpad safely releases multiple feature prototypes - in production
Edit
#ruby-on-rails
#open-source
#continuous-integration-ci
#user-experience-ux
#a-b-testing
Summarized using AI

Chanko - How Cookpad safely releases multiple feature prototypes - in production

Kenta Murata • April 23, 2012 • Austin, TX

In this presentation, Kenta Murata discusses Chanko, a framework developed at Cookpad to facilitate safe and rapid prototyping of new features in production Rails applications. The primary focus is on how Cookpad manages to test various concurrent features safely, obtain real user feedback, and ensure high-quality code without disrupting the production environment. Key points covered include:

  • Introduction to Chanko: Originally developed by Cookpad, Chanko allows for the release of experimental features to specific user segments, enabling direct feedback from real users and ensuring quick iterations based on user experience.
  • The Evolution of Development Practices: Cookpad shifted from traditional development methods, which utilized closed environments and topic branches, to a more open approach aimed at gathering real user feedback. This methodology draws inspiration from the principles outlined in 'The Lean Startup.'
  • Challenges Overcome by Chanko: The development team faced three major challenges: minimizing negative impacts from experimental code, ensuring production quality was not compromised, and maintaining a rapid development pace.
  • Chanko’s Functionality: It allows developers to manage experimental code safely, with features that can revert any errors automatically to the original production version, thus preventing user-facing issues. The presentation included a detailed live demonstration of Chanko in action, highlighting how to create and manage Chanko units effectively.
  • Practical Applications of Chanko: Throughout its use, Cookpad has managed numerous Chanko units simultaneously, continuously running around 30 to 40 at any given time while allowing for thorough testing of design layouts and features.
  • Error Management and Feature Roll-outs: Chanko includes robust mechanisms to handle errors separately from the main application, allowing the team to prioritize bugs and issues, thus ensuring a seamless user experience.

In conclusion, Chanko exemplifies an innovative approach to integrating experimentation into a high-traffic production environment. By employing Chanko, Cookpad has effectively enhanced its service development processes, allowing for continuous improvement while keeping production stable and reliable. The code is available on GitHub for others to utilize and adapt for their needs.

Chanko - How Cookpad safely releases multiple feature prototypes - in production
Kenta Murata • April 23, 2012 • Austin, TX

Chanko provides a simple framework for rapidly and safely prototyping new features in your production Rails app, and exposing these prototypes to specified segments of your user base.

With Chanko, you can release many concurrent features and independently manage which users see them. If there are errors with any chanko, it will be automatially removed, without impacting your site.

Chanko was extracted from Cookpad.com where the team uses it daily to test new features live, in production, on the largest Rails site in Japan which serves 500 million page views a month to a user based of over 15 million highly engaged uses.

RailsConf 2012

00:00:24.660 Come on in! Let me start my presentation. Thank you for coming to join my talk session. This is my first time at RailsConf, and I'm happy to be here. Thank you very much.
00:00:47.010 My name is Kenta Murata. I am a maintainer of BigDecimal and also of Protoform support. I moved from Sapporo in Japan to Tokyo about five months ago. Next Tuesday, in September, the regional Ruby conference will be held in Sapporo, which is one of the largest Ruby conferences in Japan. It’s a great place, and I hope you will all come to visit. As I mentioned, I'm originally from Sapporo, but I moved last December to join Cookpad. Cookpad is a website for sharing recipes, and it is actually the largest recipe site in Japan. Our purpose is to make people smile by making cooking more fun. We have around 15 million unique users per month, and we receive about 543 million page views monthly. Around 40 engineers work daily to provide value to these users. The site currently runs on Ruby Enterprise Edition and Rails 3.2, but we are planning to upgrade to Ruby 1.9.3 and Rails 4.2 by this summer. All our other services are hosted on AWS.
00:02:20.970 Today, I'll talk about how we develop products at Cookpad. We have recently open-sourced one of our internal tools called Chanko, which we use heavily in development. First, I will give a brief overview of what Chanko is, why it was made, and how we use it. This will be followed by a live demo and a more detailed description of Chanko's features. Finally, I will give some examples of how we use Chanko in development.
00:03:08.600 To start, we have developed our services using a conventional approach. We used topic branches to implement new features, validate hypotheses, and perform user testing. Unfortunately, we realized the limitations of developing this way about a year ago. Using this method made it difficult to evaluate the real value of the things we built, even though we were using personas and user testing. We still weren't confident that the features we were building were providing actual value. So, we decided to change the way we develop new features. We stopped working in a closed world and went out into the real world, wanting to base our new services on real feedback from our users.
00:03:59.750 Currently, we integrate new experimental services directly into the master development branch. Right from the start of development, we make these experimental features available to our internal staff and select users in the production environment. This change allows us to receive feedback from actual users based on real experiences. By constantly validating or invalidating our hypotheses based on this feedback, we can ensure that development is proceeding in the right direction.
00:04:21.739 This service development method is rooted in some ideas found in the book 'The Lean Startup.' However, combining unfinished, potentially unsafe code with production code can be challenging. Experimental code might be of low quality and may have many bugs that could potentially disrupt the user experience. If we spend all our time testing and conducting code reviews, it will slow down experimentation and reduce the number of hypotheses we can test with our user base.
00:04:51.229 To summarize the topic, we identified three problems we had to solve: First, we needed to limit the impact of any bugs or errors caused by experimental code. Second, we needed to ensure that the quality of the production code was not affected. Lastly, while maintaining the stability and quality of our job, we had to ensure that we could maintain a high speed of development.
00:05:01.909 We really needed a method to satisfy these three requirements. Unfortunately, we couldn't find existing solutions, so we decided to create our own service. We named it Chanko, and it is now available on GitHub. Chanko allows us to maintain a stable and high-quality service while continuing our experiments through trial and error.
00:05:21.469 So what exactly is Chanko? I will explain that with a simple example. This is the system that Cookpad currently uses in the production environment. With Chanko, we can provide a new feature with a different design to a limited set of users. Chanko can handle errors that occur from the new features. If an error occurs, Chanko automatically reverts to the original feature instead of raising an exception for the users.
00:07:47.200 Next, let me explain how to create a unit of Chanko from scratch. First, you use the command to generate the template of a unit. Chanko maintains a single directory for each unit called a unit. The code that you create is a file with the same name as the unit. Inside this file, you can add extensions for models or controllers, as well as stylesheets and view templates, which are maintained in independent files and stored in directories in the same way as a Rails application.
00:08:11.410 After generating the template, you write code in the main unit to specify which users are able to see the feature. In this case, we have it limited to Cookpad staff. The scope block registers a function that can be activated from a view. In this case, the partial 'recipes/new search' will be displayed only for users defined in the active block. The next step is to edit the existing view containing the content to be modified by Chanko.
00:08:58.460 This will override the existing layout that is currently being used in production. We want to activate this when the unit is active by wrapping the original HTML in an 'invoke' block. This invocation relates to the unit definition I explained earlier. These parameters are the name of the unit and the function being invoked. The application will display different layouts for generic users and for staff users. If an error occurs inside the invoked Chanko unit, the original code will execute instead, reverting back to the original layout.
00:10:37.200 This concludes my brief overview of Chanko. Next, I would like to give you a simple live demonstration of how Chanko works.
00:10:56.370 The basis for the demo is a simple blog site that can be used by multiple authors. To keep things simple, you log in using only your username and password and just the username in the session. First, I will start the server.
00:11:57.510 Oh, I'm sorry! I forgot to set up my database.
00:12:27.570 Next, I will log in and write an article.
00:12:47.939 Now I will log out and log back in as myself to write another article.
00:13:21.720 Currently, the application shows the authors' names next to the articles. But what if I would like to test showing the user avatar instead? Moreover, I'd like to test this feature only for specific users. Let's try replacing the usernames with avatars for the user whose name is Chuck Todd.
00:13:54.420 First, I generate a new Chanko unit. The unit I will create is called 'avatars.' This generates the template files I need. Next, I need to restart the server.
00:14:45.699 Now, I want to enable this unit for the username 'Chuck Todd', so I add an activity block. In this blog context, this block refers to the views from which the function is called, allowing access to all the helpers that can be used by the view.
00:15:35.810 Next, I implement a function that will return the avatar image for Chuck Todd. Now, I need to create a partial view named 'cookpad_avatar' that will be rendered within this function. But before that, I will edit the existing view of the article index to invoke the function.
00:17:15.080 In this example, I will insert an invoke method to replace the username with the avatar at the appropriate location. The original code will be encapsulated within the invoke method.
00:18:10.760 Finally, I create the new partial view to render the avatar image. For the purpose of this demo, this file simply contains an image tag to display the Cookpad logo. I've already added the Cookpad logo to the application's assets. Now the unit is ready to be deployed.
00:18:41.900 Logging in as a user, the username fields in the articles are now replaced with the Cookpad logo. As long as any other user logs in, they will see the original layout.
00:19:05.060 As I mentioned, Chanko is able to gracefully handle exceptions within units. However, by default, Chanko is set to raise exceptions in developmental mode.
00:20:06.060 This configuration can be set in the Chanko initializer. If I edit this initializer to enable Chanko to aggressively handle exceptions, then we can see how it reverts gracefully to the original layout if an exception is raised.
00:21:01.920 So that is how Chanko works. Next, let me explain in more detail how to use Chanko and what additional capabilities it offers.
00:21:18.630 First, I invoke the 'invoke' method, which is at the heart of Chanko and is used to activate new features, as I showed you in the demo. Along with the invoke method, you need to include the Chanko invoker module in the context where it is used, such as the application controller.
00:22:02.030 The invoke method can also accept a block, referred to as a default block. This default block is evaluated in two cases: when the unit is available and when there are errors occurring in a function of the unit. When you pass pairs of a unique name and function name, the availability is checked in the given context, and the first available will be evaluated.
00:22:39.600 Chanko also enables you to insert invokes inside controllers by passing the name of the controller to the scope method. This will allow, for example, a controller action to redirect to a different URL if needed when this feature is active.
00:23:13.020 Additionally, you can extend existing models using Chanko. For example, you can add new associations to instances or override methods using the models block against each other. The methods are scoped to a proxy object called the unit, which allows you to call only the methods defined in the active Chanko from the extension.
00:24:01.660 Chanko may sound similar to Rails engines, but they have very different characteristics. Chanko focuses on maintaining a stable service while extending the features of an existing application, whereas Rails engines focus on keeping features independent of the existing application.
00:24:50.450 Finally, I'd like to share some different ways we use Chanko at Cookpad. We develop services for PCs, mobile phones, and our Android app, currently working on 30 to 40 Chanko units at the same time with around 200 units developed so far.
00:25:27.210 This interface was made so Cookpad staff can toggle Chanko units on and off without restarting the application. This interface is also implemented through Chanko. At Cookpad, we use a lot of caching to provide fast and responsive service. We achieve this while also using Chanko by using a special cookie to disable page caching for specific users.
00:26:00.300 Chanko units log errors to a database. We have a special area to display errors from the main code and Chanko units separately. This way, we can prioritize resolving errors in the main code over those occurring in experimental Chanko units.
00:26:35.900 Recently, Cookpad changed the layout of its top page. We tested several different versions of the top page using Chanko. Initially, we tested several designs internally and then selected three for a limited set of users. Based on their feedback, we chose the final layout, which is now displayed by default.
00:27:05.900 Another example is the 'Sata' tool for smartphones, which was also tested using Chanko before being released. We also use Chanko for time-limited features, such as seasonal promotions, making it easy to disable them after a certain period.
00:27:59.940 If you are interested, please check out Chanko, fork it, and use it as you like. If you have any questions, feel free to tweet me or reach out to Shingo, who is the main Chanko committer. Thank you!
Kenta Murata
@mrkn
Explore all talks recorded at RailsConf 2012
+65