Search
Cameron Dutro
Kuby: Active Deployment for Rails Apps
Edit
#kubernetes
#deployment
#ruby-on-rails
#devops
#containerization
#capistrano
Summarized using AI

Kuby: Active Deployment for Rails Apps

Cameron Dutro • May 17, 2022 • Portland, OR

The video titled "Kuby: Active Deployment for Rails Apps," presented by Cameron Dutro at RailsConf 2022, addresses the challenges Rails developers face when deploying applications and introduces Kuby, a new deployment tool aimed at simplifying this process. Cameron begins by reflecting on the strengths of the Rails framework, such as its user-friendly features like Active Record and a robust directory structure, which facilitate development but fall short during deployment.

Key Points Discussed:
- Deployment Challenges: Cameron highlights the lack of an official deployment tool for Rails applications, which can make production deployment complex and daunting for developers, causing many to lose interest during this phase.
- Current Options: He reviews several deployment options available to Rails developers:
- Heroku: Popular but can become expensive as application demands grow.
- Bare Metal Solutions: Using EC2 instances with tools like Capistrano, which can be difficult to set up successfully.
- Managed Solutions: Utilizing services such as Render or Fly can incur vendor lock-in, limiting flexibility.
- Journey Through Deployment: Sharing his personal history in web development, Cameron narrates his transition from simple FTP uploads to complex solutions involving Docker and Kubernetes at GitHub.
- Introduction of Kuby: Cameron presents Kuby as a solution for active deployment, highlighting its Rails-native configuration, ease of use, and ability to deploy applications directly to Kubernetes clusters via straightforward commands. Kuby supports major cloud vendors and provisions key features like SSL certificates, asset servers, and database instances, aiming to bridge the gap left by existing tools.
- Kuby's Features: Kuby allows Rails apps to be packaged into Docker images, automatically sets up environment configurations, and manages scaling, making it a robust option for developers looking to streamline their deployment process. The tool’s plugin system further enhances its capabilities for scaling up applications with ease.

In conclusion, Cameron emphasizes the need for an active deployment solution in Rails, pointing out that while frameworks like Active Record aid in data management, Rails lacks a dedicated deployment counterpart. He encourages the Rails community to explore and contribute to Kuby, ensuring a more manageable deployment strategy that matches the framework's development agility. The talk wraps up with a Q&A session that addresses local testing with Kuby and discusses future enhancements for the tool.

Kuby: Active Deployment for Rails Apps
Cameron Dutro • May 17, 2022 • Portland, OR

One of the Rails mantras is "convention over configuration," sane defaults that limit the cognitive overhead of application development. It's easy to learn and easy to build with... right up until you want to deploy your app to production. At that point, the hand-holding ends. Like the Roadrunner, Rails stops right before the cliff and lets you, Wile E. Coyote, sail over the edge. We have active record for interacting with databases, active storage for storing files, etc, but where's active deployment? Come learn how Kuby, a new deployment tool, is trying to bridge the gap.

RailsConf 2022

00:00:12.620 I am Cameron Dutro, and today I am giving a talk on what I'm calling Active Deployment for Rails apps.
00:00:19.440 First of all, welcome, and thank you for coming. This is amazing! I think this is the third talk I've given at RailsConf, but I haven't given one in four or five years, so I'm feeling the nerves right now. I appreciate all of you smiling behind your covered faces.
00:00:31.320 So, who am I? I am Cameron Dutro, and I work at GitHub. A couple of my coworkers are sitting in the very front. Thanks, guys! My pronouns are he/him/his. I am the dad to two adorable goofballs, Hallie and Mona. It's actually quite a miracle that I get to be here today because both of them are under three.
00:00:43.440 My heroic wife told me, 'Hey, you got a talk accepted at RailsConf; go ahead and go, and I'll take care of them.' So, if we could give a round of applause for Kelly, that would be amazing!
00:01:06.180 I’m also a huge Seahawks fan, so please come up and talk to me about how terrible they’re going to be this year.
00:01:22.100 We all know that Rails is awesome. We've been told numerous times, and we all get it. We’re all sitting here today because Rails is really easy to get into. Active Record makes it easy to access the database layer, and development is super fast and easy. Rails provides a fantastic directory structure, ensuring there is the right place for every file. We have tons of community-provided libraries, and in general, the community is fantastic.
00:01:49.259 And if you listen to David, you know that it’s also great because it's not Java! However, the problem, as I see it, comes when it's time to deploy.
00:02:03.780 The unfortunate truth is that Rails has no official tool or guide to aid in deploying an app to production. In other words, there’s no active record for deploying your app.
00:02:22.860 In 2019, Stefan Vintumaya appeared on the Ruby Rogues podcast, episode number 403, and made a couple of observations that I think are really important and have guided my efforts toward remedying this in the Rails world. He stated, 'In my experience, deployment is one of the major problems for normal Rails users. It's a big pain to set up a deployment system for a Rails application, and I don't see anything out there that makes it easier.'
00:02:43.680 Rails is sort of famous for having a good learning curve. He also mentioned that he believes we lose quite a lot of companies and new developers on this step because everything else is so much easier with Rails, but that last step, which is super important, remains complicated.
00:03:20.580 So, what are our options for deploying a Rails app? Well, a lot of us love Heroku. I’m not here to tell you that Heroku is bad; Heroku has its merits. However, it can get pretty expensive over time. You add more dynos, your database size increases, and you start to pay more and more.
00:03:50.000 You could, of course, go the bare metal route, getting an EC2 instance in AWS and using Capistrano. That’s difficult to set up, though, even though Capistrano is a wonderful tool; it can be challenging to set it up effectively, and I always get it wrong the first time.
00:04:14.939 You could also use something like an EC2 instance with Chef, Puppet, or Ansible, but there is a lot to learn there. There’s a whole separate community around deployment and setting up servers using these tools. And lastly, you could use a hosted solution like Render, Fly, Cloud 66, or again, Heroku.
00:04:40.139 These are great tools as well, but there’s a lot of vendor lock-in. That’s one of the nice things about Capistrano; it’s all your own code, and it’s all your own infrastructure. There’s no worrying about vendor lock-in. You can take your Heroku scripts or tasks to another provider and use them without any issues.
00:05:05.460 Recently, DHH and the Rails Core team have changed their website to state that Rails scales from Hello World to IPO, which is great. But I would love to see somebody get to an IPO without being able to deploy their app to production.
00:05:35.820 Let’s talk about our journey today. We’re going to discuss what deployment is, what’s changed in the way we, as web developers, deploy apps over the last 20 years, what we’re currently using as Rails developers for deployment, and at the end of the talk, I want to introduce a tool that I’ve created called Kubi, which I also want to position as active deployment for Rails.
00:06:05.100 So, what is deployment? Well, you build a great website and want other people to see it. What do you need to make that happen? First, you need a computer, potentially a server—hopefully one that someone else runs and maintains. You also need some software that listens for requests, invokes your code, and sends back the response to the browser.
00:06:21.960 You might also need a domain name, which involves going to some registrar and getting something like mycoolsite.com. Additionally, you might need some persistent storage, like a database. This is all pretty standard, and I think most of us would intuitively understand that these are the things you would need to put a website onto the internet for others to see.
00:06:50.280 I’d like to talk about my deployment journey and how I got to where I am and why I’m thinking about these things now. My first website was a hand-coded HTML site I built in 2002, called CCM Glass. It was inspired by the glass my siblings and I found on beaches, and I wanted to create a business to sell it.
00:07:23.280 I didn’t have a computer at the time, but when my cousin came to visit, I borrowed his computer and spent afternoons typing in examples from an HTML book I had brought. Watching my website come to life as I hit refresh was amazing! Unfortunately, CCM Glass never got a single order because I didn’t build any order capability—there was no backend code, just HTML.
00:07:53.460 This was deployed simply by copying and pasting the files onto an FTP server using a free hosting provider I found back then. Fast forward a couple of years, from 2004 to 2010, I picked up PHP.
00:08:05.460 PHP was a lot more dynamic; it allowed you to store and retrieve data from MySQL. I created a calendar app for my high school to display the upcoming events, which was also created in PHP and stored its data in a MySQL database. This was pre-object-oriented PHP, so it was just a bunch of function calls.
00:08:34.680 Once again, this was deployed via FTP. Talking about dragging files off your file system and using something like FileZilla to make the site live. Fast forward a few more years; in 2010, I got my first real job at Flutter.
00:09:01.200 This was my first job out of college, working on a Django app which had a MySQL database. We deployed it using Capistrano. That was a bit strange, as the shop was a Python shop but we used a Ruby tool to deploy.
00:09:18.060 I think this illustrates that not much attention was being paid to deployment in the Python world at the time. The Ruby world also seemed limited; we really only had Capistrano as a deployment tool back then. I remember a boss explaining what a Ruby symbol was, and we both chuckled wondering why we didn’t just use strings.
00:09:32.460 Fast forward another year as Flutter was acquired by Twitter, I discovered how deployment works at scale. At that time, Twitter had thousands of app servers, and they had used Capistrano for a long time as it was a Rails app, but this eventually became untenable.
00:10:01.200 They had to create their own deployment tool called 'Murder,' aptly named with references to crows, as everything at Twitter had a bird name. This deploy tool interestingly used the BitTorrent protocol. The first server would seed the version, then all other servers would pull from each other until they all had a copy of the code.
00:10:25.620 This method reduced the pressure on the Git server since thousands of app servers weren't trying to pull from the Git server simultaneously. Instead, they communicated among themselves, resulting in much quicker deployments and reduced server load.
00:10:58.740 As time passed, Twitter moved to Mesos, a system that predates Kubernetes, and it focuses on clusters rather than individual servers like Capistrano.
00:11:28.920 Mesos allowed them to manage various services separately without tightly coupling them to specific resources on the server. Fast forward another year, and I was working at Lumos Labs, initially starting with VPSs and a bespoke Capistrano setup.
00:11:58.740 In 2016, we transitioned to using Docker and Amazon’s Elastic Container Service (ECS), and finally, in 2017, we switched to Kubernetes. Currently, at GitHub, we primarily use Kubernetes with a chat ops system running in Slack.
00:12:38.100 Kubernetes abstracts away the details for developers. It allows you to run commands and deploy applications with ease. The focus on simplifying the deployment experience is essential since Kubernetes is complex; even after years of working with it, I know only 10% of it.
00:13:06.720 Let’s talk about why Kubernetes is beneficial for active deployment. First, Kubernetes offers a true platform. We don’t need to set up those systems ourselves; it’s declarative. You tell Kubernetes what you want, and it figures out how to do it for you.
00:13:35.460 Kubernetes is portable, meaning you can deploy to any cloud server using the same configuration with very few changes. It’s cluster-focused and treats your cluster as a single resource.
00:14:05.100 Kubernetes is also resilient, automatically restarting failed processes when necessary. Lastly, it is scalable; you can adjust the number of web workers or Sidekiq workers based on demand.
00:14:39.900 Now, I want to introduce Kubi. Kubi is an active deployment setup for Rails apps. It packages up and deploys your Rails app into a Docker image, which is then deployed to a Kubernetes cluster.
00:15:10.080 Kubi aims for minimal and easy-to-read configuration. It supports major cloud vendors such as Digital Ocean, Linode, AWS EKS, and Azure. Kubi is Rails native, allowing you to install it like any gem in your Gemfile.
00:15:43.500 It automatically manages secrets and credentials, standing up a database instance for you. Additionally, it creates a static asset server, ensuring assets are served via NGINX, and acquires SSL/TLS certificates for your domain automatically using Let's Encrypt.
00:16:02.760 Your configuration for Kubi is straightforward. First, you define your application environment; then, inside the Docker section, specify your credentials for talking to your Docker registry.
00:16:32.400 You would have an image URL where the built Docker image would be tagged and pushed. The Kubernetes section allows you to add the Rails app plugin. Most functionality in Kubi is developed via plugins.
00:17:07.560 How does deployment work in Kubi? The first deploy involves a command to set up the environment, installing operators, Helm charts, and anything else needed.
00:17:36.000 Next, the build command processes your Rails application into a Docker image. Then, the push command sends that image to the specified registry, and finally, the deploy command executes the deployment of the newly built image into the cluster.
00:17:58.560 One of the fantastic features of Kubi is its robust plugin system. For instance, you can stand up your own Redis instances in your cluster or set up Sidekiq alongside Redis with minimal configuration.
00:18:23.480 Kubi allows you to run a copy of your app for each pull request, thanks to a dedicated gem. There’s also a community-contributed gem that deploys the Go-based RPC components for Action Cable into your cluster.
00:18:45.300 To add Sidekiq, for example, you would simply add the respective gem to your Gemfile, configure it as needed, and then follow a similar process to set it up as you would for any other feature.
00:19:07.080 With Kubi, you have access to many enhancements that elevate your Rails deployment.
00:19:29.280 The platform that Kubi operates on enables you to utilize the container scheduling capabilities of Kubernetes, as well as its networking, storage API, and role-based access control.
00:19:50.160 Overall, Kubi is designed to simplify the deployment process using Kubernetes, making it accessible and efficient for Rails developers. For more information, you can visit getcubi.io and explore the documentation.
00:20:20.640 Thank you for your attention! I would love to see more people hop on board to help out with this project. If you're interested, visit getcubi.io for all the documentation, and let’s get deploying apps with Kubi!
00:21:05.340 I have about a minute and 54 seconds for questions. If there are any, feel free!
00:21:10.500 In the back there, the question is: 'Can you test a deployment locally using Minikube on your laptop?' Yes, you can set up a bare-metal provider for that. You can also use a tool called Kind, which is a local cluster runner.
00:21:31.200 It uses Docker to spin up a whole cluster on your locally running Docker. Information about this can be found on GitHub under kubi-kind.
00:21:56.340 Regarding the next steps for Kubi, the sky's the limit. I want to implement features and ensure that there’s a good upgrade story. It’s currently in a pre-alpha state and needs enhancements.
00:22:20.100 Lastly, regarding how Kubi supports migrations, it currently runs DB migrate automatically each time you deploy a new version. However, we may consider improvements for handling not-backwards-compatible migrations.
00:22:47.700 That’s all the time I have. Thank you, everyone!
Cameron Dutro
@camertron
Explore all talks recorded at RailsConf 2022
+68