00:00:12.620
I am Cameron Dutro, and today I am giving a talk on what I'm calling Active Deployment for Rails apps.
00:00:19.440
First of all, welcome, and thank you for coming. This is amazing! I think this is the third talk I've given at RailsConf, but I haven't given one in four or five years, so I'm feeling the nerves right now. I appreciate all of you smiling behind your covered faces.
00:00:31.320
So, who am I? I am Cameron Dutro, and I work at GitHub. A couple of my coworkers are sitting in the very front. Thanks, guys! My pronouns are he/him/his. I am the dad to two adorable goofballs, Hallie and Mona. It's actually quite a miracle that I get to be here today because both of them are under three.
00:00:43.440
My heroic wife told me, 'Hey, you got a talk accepted at RailsConf; go ahead and go, and I'll take care of them.' So, if we could give a round of applause for Kelly, that would be amazing!
00:01:06.180
I’m also a huge Seahawks fan, so please come up and talk to me about how terrible they’re going to be this year.
00:01:22.100
We all know that Rails is awesome. We've been told numerous times, and we all get it. We’re all sitting here today because Rails is really easy to get into. Active Record makes it easy to access the database layer, and development is super fast and easy. Rails provides a fantastic directory structure, ensuring there is the right place for every file. We have tons of community-provided libraries, and in general, the community is fantastic.
00:01:49.259
And if you listen to David, you know that it’s also great because it's not Java! However, the problem, as I see it, comes when it's time to deploy.
00:02:03.780
The unfortunate truth is that Rails has no official tool or guide to aid in deploying an app to production. In other words, there’s no active record for deploying your app.
00:02:22.860
In 2019, Stefan Vintumaya appeared on the Ruby Rogues podcast, episode number 403, and made a couple of observations that I think are really important and have guided my efforts toward remedying this in the Rails world. He stated, 'In my experience, deployment is one of the major problems for normal Rails users. It's a big pain to set up a deployment system for a Rails application, and I don't see anything out there that makes it easier.'
00:02:43.680
Rails is sort of famous for having a good learning curve. He also mentioned that he believes we lose quite a lot of companies and new developers on this step because everything else is so much easier with Rails, but that last step, which is super important, remains complicated.
00:03:20.580
So, what are our options for deploying a Rails app? Well, a lot of us love Heroku. I’m not here to tell you that Heroku is bad; Heroku has its merits. However, it can get pretty expensive over time. You add more dynos, your database size increases, and you start to pay more and more.
00:03:50.000
You could, of course, go the bare metal route, getting an EC2 instance in AWS and using Capistrano. That’s difficult to set up, though, even though Capistrano is a wonderful tool; it can be challenging to set it up effectively, and I always get it wrong the first time.
00:04:14.939
You could also use something like an EC2 instance with Chef, Puppet, or Ansible, but there is a lot to learn there. There’s a whole separate community around deployment and setting up servers using these tools. And lastly, you could use a hosted solution like Render, Fly, Cloud 66, or again, Heroku.
00:04:40.139
These are great tools as well, but there’s a lot of vendor lock-in. That’s one of the nice things about Capistrano; it’s all your own code, and it’s all your own infrastructure. There’s no worrying about vendor lock-in. You can take your Heroku scripts or tasks to another provider and use them without any issues.
00:05:05.460
Recently, DHH and the Rails Core team have changed their website to state that Rails scales from Hello World to IPO, which is great. But I would love to see somebody get to an IPO without being able to deploy their app to production.
00:05:35.820
Let’s talk about our journey today. We’re going to discuss what deployment is, what’s changed in the way we, as web developers, deploy apps over the last 20 years, what we’re currently using as Rails developers for deployment, and at the end of the talk, I want to introduce a tool that I’ve created called Kubi, which I also want to position as active deployment for Rails.
00:06:05.100
So, what is deployment? Well, you build a great website and want other people to see it. What do you need to make that happen? First, you need a computer, potentially a server—hopefully one that someone else runs and maintains. You also need some software that listens for requests, invokes your code, and sends back the response to the browser.
00:06:21.960
You might also need a domain name, which involves going to some registrar and getting something like mycoolsite.com. Additionally, you might need some persistent storage, like a database. This is all pretty standard, and I think most of us would intuitively understand that these are the things you would need to put a website onto the internet for others to see.
00:06:50.280
I’d like to talk about my deployment journey and how I got to where I am and why I’m thinking about these things now. My first website was a hand-coded HTML site I built in 2002, called CCM Glass. It was inspired by the glass my siblings and I found on beaches, and I wanted to create a business to sell it.
00:07:23.280
I didn’t have a computer at the time, but when my cousin came to visit, I borrowed his computer and spent afternoons typing in examples from an HTML book I had brought. Watching my website come to life as I hit refresh was amazing! Unfortunately, CCM Glass never got a single order because I didn’t build any order capability—there was no backend code, just HTML.
00:07:53.460
This was deployed simply by copying and pasting the files onto an FTP server using a free hosting provider I found back then. Fast forward a couple of years, from 2004 to 2010, I picked up PHP.
00:08:05.460
PHP was a lot more dynamic; it allowed you to store and retrieve data from MySQL. I created a calendar app for my high school to display the upcoming events, which was also created in PHP and stored its data in a MySQL database. This was pre-object-oriented PHP, so it was just a bunch of function calls.
00:08:34.680
Once again, this was deployed via FTP. Talking about dragging files off your file system and using something like FileZilla to make the site live. Fast forward a few more years; in 2010, I got my first real job at Flutter.
00:09:01.200
This was my first job out of college, working on a Django app which had a MySQL database. We deployed it using Capistrano. That was a bit strange, as the shop was a Python shop but we used a Ruby tool to deploy.
00:09:18.060
I think this illustrates that not much attention was being paid to deployment in the Python world at the time. The Ruby world also seemed limited; we really only had Capistrano as a deployment tool back then. I remember a boss explaining what a Ruby symbol was, and we both chuckled wondering why we didn’t just use strings.
00:09:32.460
Fast forward another year as Flutter was acquired by Twitter, I discovered how deployment works at scale. At that time, Twitter had thousands of app servers, and they had used Capistrano for a long time as it was a Rails app, but this eventually became untenable.
00:10:01.200
They had to create their own deployment tool called 'Murder,' aptly named with references to crows, as everything at Twitter had a bird name. This deploy tool interestingly used the BitTorrent protocol. The first server would seed the version, then all other servers would pull from each other until they all had a copy of the code.
00:10:25.620
This method reduced the pressure on the Git server since thousands of app servers weren't trying to pull from the Git server simultaneously. Instead, they communicated among themselves, resulting in much quicker deployments and reduced server load.
00:10:58.740
As time passed, Twitter moved to Mesos, a system that predates Kubernetes, and it focuses on clusters rather than individual servers like Capistrano.
00:11:28.920
Mesos allowed them to manage various services separately without tightly coupling them to specific resources on the server. Fast forward another year, and I was working at Lumos Labs, initially starting with VPSs and a bespoke Capistrano setup.
00:11:58.740
In 2016, we transitioned to using Docker and Amazon’s Elastic Container Service (ECS), and finally, in 2017, we switched to Kubernetes. Currently, at GitHub, we primarily use Kubernetes with a chat ops system running in Slack.
00:12:38.100
Kubernetes abstracts away the details for developers. It allows you to run commands and deploy applications with ease. The focus on simplifying the deployment experience is essential since Kubernetes is complex; even after years of working with it, I know only 10% of it.
00:13:06.720
Let’s talk about why Kubernetes is beneficial for active deployment. First, Kubernetes offers a true platform. We don’t need to set up those systems ourselves; it’s declarative. You tell Kubernetes what you want, and it figures out how to do it for you.
00:13:35.460
Kubernetes is portable, meaning you can deploy to any cloud server using the same configuration with very few changes. It’s cluster-focused and treats your cluster as a single resource.
00:14:05.100
Kubernetes is also resilient, automatically restarting failed processes when necessary. Lastly, it is scalable; you can adjust the number of web workers or Sidekiq workers based on demand.
00:14:39.900
Now, I want to introduce Kubi. Kubi is an active deployment setup for Rails apps. It packages up and deploys your Rails app into a Docker image, which is then deployed to a Kubernetes cluster.
00:15:10.080
Kubi aims for minimal and easy-to-read configuration. It supports major cloud vendors such as Digital Ocean, Linode, AWS EKS, and Azure. Kubi is Rails native, allowing you to install it like any gem in your Gemfile.
00:15:43.500
It automatically manages secrets and credentials, standing up a database instance for you. Additionally, it creates a static asset server, ensuring assets are served via NGINX, and acquires SSL/TLS certificates for your domain automatically using Let's Encrypt.
00:16:02.760
Your configuration for Kubi is straightforward. First, you define your application environment; then, inside the Docker section, specify your credentials for talking to your Docker registry.
00:16:32.400
You would have an image URL where the built Docker image would be tagged and pushed. The Kubernetes section allows you to add the Rails app plugin. Most functionality in Kubi is developed via plugins.
00:17:07.560
How does deployment work in Kubi? The first deploy involves a command to set up the environment, installing operators, Helm charts, and anything else needed.
00:17:36.000
Next, the build command processes your Rails application into a Docker image. Then, the push command sends that image to the specified registry, and finally, the deploy command executes the deployment of the newly built image into the cluster.
00:17:58.560
One of the fantastic features of Kubi is its robust plugin system. For instance, you can stand up your own Redis instances in your cluster or set up Sidekiq alongside Redis with minimal configuration.
00:18:23.480
Kubi allows you to run a copy of your app for each pull request, thanks to a dedicated gem. There’s also a community-contributed gem that deploys the Go-based RPC components for Action Cable into your cluster.
00:18:45.300
To add Sidekiq, for example, you would simply add the respective gem to your Gemfile, configure it as needed, and then follow a similar process to set it up as you would for any other feature.
00:19:07.080
With Kubi, you have access to many enhancements that elevate your Rails deployment.
00:19:29.280
The platform that Kubi operates on enables you to utilize the container scheduling capabilities of Kubernetes, as well as its networking, storage API, and role-based access control.
00:19:50.160
Overall, Kubi is designed to simplify the deployment process using Kubernetes, making it accessible and efficient for Rails developers. For more information, you can visit getcubi.io and explore the documentation.
00:20:20.640
Thank you for your attention! I would love to see more people hop on board to help out with this project. If you're interested, visit getcubi.io for all the documentation, and let’s get deploying apps with Kubi!
00:21:05.340
I have about a minute and 54 seconds for questions. If there are any, feel free!
00:21:10.500
In the back there, the question is: 'Can you test a deployment locally using Minikube on your laptop?' Yes, you can set up a bare-metal provider for that. You can also use a tool called Kind, which is a local cluster runner.
00:21:31.200
It uses Docker to spin up a whole cluster on your locally running Docker. Information about this can be found on GitHub under kubi-kind.
00:21:56.340
Regarding the next steps for Kubi, the sky's the limit. I want to implement features and ensure that there’s a good upgrade story. It’s currently in a pre-alpha state and needs enhancements.
00:22:20.100
Lastly, regarding how Kubi supports migrations, it currently runs DB migrate automatically each time you deploy a new version. However, we may consider improvements for handling not-backwards-compatible migrations.
00:22:47.700
That’s all the time I have. Thank you, everyone!