00:00:10.400
Um, yes, so let's go back one.
00:00:14.240
That's me, Donal McBreen. I'm a programmer.
00:00:16.279
I work at 37signals, and today, I'm going to discuss Kamal 2, which is the tool we're currently using to deploy.
00:00:20.560
So, I'm going to talk a little bit about Kamal in general, how it works,
00:00:22.720
and then more specifically about what we've added for Kamal 2.
00:00:25.000
We sort of have a history already, so we'll go through this. I won't take too long.
00:00:29.119
At the start, we were running a bunch of applications in AWS on their managed Kubernetes service, EKS, and we wanted to bring them in-house to run on our own hardware.
00:00:36.680
So, we needed a way to run them. One option would have been to start using Kubernetes locally, but we didn't really fancy that.
00:00:41.520
It looked quite tricky, and we were already using Basecamp, our biggest application, which had always run on our own hardware.
00:00:45.840
We were just using Capistrano to deploy it, and when we looked at what we were doing in AWS, we decided that we didn't really like this Kubernetes part.
00:00:50.559
Kubernetes looked a bit tricky, and we didn't really need it, but the Docker part looked great because we wouldn't need to artisanally manage our servers anymore, ensuring we could compile the latest Ruby and make sure all the gems installed correctly.
00:01:03.120
So, we decided we needed Capistrano for containers. If you don't know what Capistrano is, back in the days before it,
00:01:07.720
you would deploy your applications by running some script you had made up or by SSHing into a server and just typing some random commands, hoping that it worked.
00:01:19.680
Capistrano, which was originally built by Jamus Buck, who is actually talking right now in the other room, was a way to make this simpler.
00:01:29.680
The idea was that you would SSH, and Capistrano would SSH to your servers for you and have a bunch of recipes that it would run to deploy your app, allowing for repeatable builds.
00:01:43.159
That was what we decided we needed.
00:01:45.799
Kamal is kind of a modern take on Capistrano but decided to run with Docker. It's a Ruby gem built on top of SSHKit.
00:01:58.960
SSHKit is a library extracted from Capistrano that helps you run commands over SSH concurrently and wait for them all to return.
00:02:02.399
The idea is that you've got some Linux servers running Docker or about to run Docker that you want to deploy to.
00:02:12.240
These were the three big goals we had. First, it would be zero downtime. If you do a very naive version of just running Docker,
00:02:15.360
your application will boot up, sitting there on port 80, and then you've got a new version of the application, and what you need to do is take the first one down and bring the other one up.
00:02:30.240
There is a big gap during that time where you're serving error pages, which we wanted to avoid.
00:02:35.000
The second aspect was that it would be an imperative tool.
00:02:41.760
What this means is that we're going to run a command and then wait, and when the command is finished, we're done. There's nothing else going on.
00:02:57.520
Kubernetes is more declarative, and the distinction here is imperative means 'I insist,' while declarative means 'I suggest.'
00:03:08.000
With this, we're saying 'I insist you deploy,' and you tell me whether you've deployed or not.
00:03:12.240
Finally, if we're going to wait for a while to complete, we need to be as fast as possible, so we're going to be deploying Docker images.
00:03:29.439
You have to build them and move them around.
00:03:31.440
If that takes a long time, the process will take a long time, and we wanted Kamal to be as fast as possible and avoid adding any extra overhead, if possible.
00:03:35.000
So, let's move on to the prerequisites.
00:03:38.760
To run Kamal on your app, you'll need an app with a Dockerfile so you can build your Docker image.
00:03:46.480
It doesn't have to be Rails; it can be any kind of web app packaged up as a Docker image.
00:04:00.000
As David mentioned earlier, you do need a Docker registry right now. We have some ideas on how to get rid of this part, as it's quite annoying, but for now, it is necessary.
00:04:06.360
Finally, you need a bunch of servers that you have SSH access to.
00:04:09.480
If you want to install it with Rails 8, it will come installed already unless you add the skip command option.
00:04:14.080
If you want to try it out on an existing application, you can install the gem or include it in your bundle.
00:04:27.200
Then, run `kamal setup`, which will create files. The main file you need here is `config/deployed.yml`.
00:04:35.080
You'll need to specify the servers you are deploying to, details regarding your registry, and which architecture you want to build for.
00:04:47.440
Here's our simple infrastructure: you have a host from which you are deploying—this could be your laptop or a specific server.
00:04:56.000
You also have a host to which you want to deploy. You will talk over SSH, and it can do all the work to deploy in parallel if that's the case.
00:05:05.640
However, you will need your own load balancer somewhere to balance things out over those hosts, and we will just demonstrate using one host here.
00:05:21.040
We have our deployment host, application host, and Docker registry for now.
00:05:26.440
When you run `kamal setup`, you'll see the standard SSHKit output.
00:05:36.120
This is handy because you can inspect it after a deployment to see exactly what Kamal did. This heritage from Capistrano is very useful.
00:05:49.120
The first thing Kamal will do is check on the application host to see if Docker is installed. If it isn't, it will try to install it.
00:05:53.679
If it cannot be installed, you'll need to resolve that issue manually.
00:05:57.280
Otherwise, Kamal will get Docker installed, perform a Docker build on your deployment host, and push it.
00:06:02.480
Then, Kamal will SSH to your application host and perform a Docker pull, ensuring our container is present.
00:06:11.040
Next is the deployment phase. We'll focus on the application host for that.
00:06:18.400
We will start with commands coming in over SSH from our deployment host. Docker should be installed, and we have copied our image over.
00:06:30.160
The first thing we do is boot up a proxy, specifically the instance of Kamal Proxy, which is the new proxy we've built.
00:06:36.240
My colleague Kevin is giving a talk on that after this, so if you're interested in learning more, come along to that talk.
00:06:48.960
Once we boot up the application container, we tell the proxy to send traffic through to it, and now we are done, with service requests being directed to our application.
00:07:01.040
The next phase involves deploying a second version of your applications, where you've made your changes and committed them. You'll then run `kamal deploy`.
00:07:10.240
'Kamal setup' is the initial setup, and the deployment is the next phase. Initially, it will build the image, push it, and pull it onto the application host.
00:07:22.240
Now, this is where we left things off after our last deployment.
00:07:26.480
We will boot up a container running the second version of the application and then instruct Kamal Proxy to send traffic over there.
00:07:35.800
And we're done. This is a simple overview of how it works.
00:07:41.399
There are additional features for job servers where you don’t need a proxy, and you can configure those separately.
00:07:57.679
One interesting aspect to discuss is what we call asset bridging.
00:08:10.880
With the naive version of deployment, if you are running multiple application hosts and they deploy at different rates, you can encounter issues.
00:08:25.199
You may have new containers with updated assets and old containers still serving from older assets, which can lead to 404 errors and unstyled pages, making the deployment visible to users.
00:08:35.000
This is not desirable; we want zero downtime and to make the deployment as invisible as possible.
00:08:49.040
This is a common problem, and many rails apps take steps beforehand, like copying assets to an S3 bucket.
00:09:03.200
However, with Kamal, we want to ensure you don't have to worry about this.
00:09:14.720
For example, when you create a new Rails application, you'll find a line in the config that sets the asset path to Rails public assets.
00:09:23.520
Kamal will look for the assets and, during the build process, will boot up the application image but will change the entry point just to sleep, allowing it to remain idle.
00:09:35.880
Afterward, it will copy those assets into a specific folder on the application host, outside of Docker.
00:09:44.760
Then, when you deploy again, you’ll have the assets from the last deployment and the new ones you just collected.
00:09:55.840
By combining the old and new assets, you ensure that you won't face 404 errors; the application will have access to both sets of assets.
00:10:04.560
This fixes the issue where your old containers might still be serving to users.
00:10:13.280
You can map those assets back to public assets when you boot the new version of the application, ensuring a seamless transition.
00:10:22.520
When you boot up the new version of your application, the public assets will contain both the old and new assets.
00:10:34.080
This means you won't have to worry about which container the requests hit, thanks to this asset bridging feature.
00:10:45.440
The result is clear: you avoid 404 errors during deployments, and you don't need to rely on a CDN or create special asset hosts.
00:10:56.640
And yeah, that is asset bridging.
00:11:05.760
Now, just to confirm, it is releasing today, but it has not been released as of yet.
00:11:12.560
I decided it would be better to present this talk before the actual release to ensure everything is prepared and ready for it.
00:11:24.720
This means the documentation is not up to date yet; however, it will be updated later today.
00:11:38.560
So, what's new? One of the biggest changes is that we are switching the proxy we use.
00:11:45.840
In Kamal 1, we used Traefik as this proxy to switch between containers.
00:11:53.440
Traefik is an open-source application proxy designed to configure itself automatically and dynamically.
00:12:01.720
We would utilize it by booting our container and noticing the labels we set, allowing it to direct traffic based on those labels.
00:12:14.400
However, this created issues. The main problem was a mismatch between the declarative and imperative approaches.
00:12:22.800
With Kamal, we are insisting on specific commands, but Traefik works by taking suggestions.
00:12:31.920
You would need to boot the container with labels and wait for Traefik to notice them, which led to discrepancies.
00:12:46.560
This made draining requests very difficult because container labels are immutable.
00:12:52.800
You could tell Traefik to send traffic to a container, but you couldn't stop it from doing so.
00:13:05.600
This resulted in situations where we had to use hacks to manipulate the health check so Traefik would stop sending traffic.
00:13:16.320
It wasn't great, and we often faced difficulties understanding errors because Traefik was a general-purpose proxy with its complex language.
00:13:28.880
As a result, we decided to build our own proxy tailored for Kamal, which is zero-configuration and very straightforward.
00:13:41.840
With the new proxy, we can run multiple applications without boot time configuration, keeping it simple.
00:13:53.440
The command structure in Kamal is straightforward, where deploying is just a matter of sending a command that follows a one-to-one mapping.
00:14:07.080
When you type `kamal deploy`, it goes to the proxy, which confirms the action—we say we’ve deployed, and the entire process is smooth.
00:14:18.080
Everything is streamlined and quick because you are not polling and waiting for responses.
00:14:30.160
One of our goals is to simplify deployment, especially for users who may struggle with setup.
00:14:40.840
We added a command that can open a Rails console easily. `app exec` is a Kamal command that either boots a new container or execs into an existing one.
00:14:53.040
You can pass in `rails console`, which provides ready access to a running Rails console in production.
00:15:05.080
However, if users have specific requirements, they may want to run the console on a certain host. Aliases allow for this, letting you specify commands on an app level.
00:15:15.360
So, after setting up those aliases, running `kamal console` would launch the console directly.
00:15:24.680
In conclusion, we built Kamal 1 with a specific use case in mind, but Kamal 2 aims to streamline the process further.
00:15:35.040
Now, with Kamal 2, you're just typing `rails new`, and if you've got your domain and a $6 digital ocean droplet, you can get running on HTTPS as quickly as possible.
00:15:41.440
We appreciate your interest and engagement with Kamal and look forward to your feedback on these new functionalities.
00:15:54.920
As a final note, the Kamal 2 release will happen later today, and additional updates will be provided in the documentation at that time.
00:16:03.440
Feel free to ask any questions or provide insights about these changes. Thank you for your attention.
00:16:15.520
We hope you enjoy using Kamal!
