Talks
Panel: Rails Deployment
Panel: Rails Deployment
by Paweł Pacana, Michał Łomnicki, and Tymon TobolskiThe video titled 'Panel: Rails Deployment' features a discussion among Tymon Tobolski, Paweł Pacana, and Michał Łomnicki at the wroc_love.rb 2016 event. The main theme revolves around the comparison of cloud hosting versus dedicated hosting for application deployment. The speakers argue the merits and drawbacks of both hosting options based on their experiences, emphasizing the importance of developer convenience and performance.
Key Points Discussed:
- Hosting Types:
- Dedicated hosting offers potential performance benefits but requires more maintenance and expertise, making it less ideal for developers focused on application features.
- Cloud hosting provides abstraction over hardware failures, reducing the burden of management tasks like RAID setup.
Personal Experience:
- Paweł shares his initial struggles with Heroku, highlighting the operational challenges faced when self-managing servers. He notes a turning point where he prioritized ease of development over the complexities of managing servers.
- They transitioned internal applications from DigitalOcean to Heroku, which simplified scaling and resource management as the applications grew.
Alternative Solutions:
- The panel introduces Dokku as a lightweight alternative to Heroku, allowing developers to also manage staging environments on dedicated servers at lower costs.
Deployment Processes:
- They discuss Git-based deployment strategies and the possible use of tools like Capistrano and Mina, noting the importance of simplifying deployment tasks while still keeping necessary dependencies in mind.
Configuration and Management Tools:
- Discussion on configuration management tools such as Ansible, Chef, and Puppet, stressing the importance of familiarity and finding tools that fit the context of a project.
Continuous Deployment:
- Emphasis on continuous deployment practices, the need for regular backups, and strategies for database migrations while avoiding redundancy across configurations to maintain consistency.
Scaling and Monitoring:
- Strategies for handling application scaling, monitoring usage patterns, and ensuring effective allocation of resources to prevent bottlenecks.
Conclusions and Takeaways:
- The panel concludes that while dedicated hosting can be beneficial, starting with cloud solutions like Heroku makes deployments smoother for developers, allowing for scalability without excessive maintenance. Frequent deployments, backed by robust testing and rollback strategies, should be encouraged to achieve optimal performance and ensure application integrity.
00:00:15.430
So here's Paweł Pacana from our team, Tymon Tobolski, and anyone else.
00:00:21.640
Today we will discuss cloud versus dedicated hosting.
00:00:33.210
My idea is that you might want to use dedicated hosting when you're focused on performance.
00:00:40.600
Dedicated hosting and bare metal servers might provide you some performance benefits over cloud solutions.
00:00:52.000
However, I'm not sure if I would recommend starting with dedicated servers because it means you'll have to worry about the hardware.
00:01:06.490
Cloud hosting is nice because it gives you an abstraction over hardware, so you don’t have to deal with failed disks or set up RAID.
00:01:13.180
Using dedicated servers might require more work from you compared to cloud, where things are sorted out already.
00:01:27.060
In my opinion, I would always start with cloud.
00:01:35.729
Of course, if you have a dedicated administrator who can manage this stuff, then okay, you can consider dedicated servers.
00:01:40.750
However, since we are mostly developers, we should focus on development rather than on the IT side of things.
00:01:53.590
For example, I used to hate Heroku because I thought I could set everything up myself. I wondered why I would pay Heroku to handle it.
00:02:05.950
However, I had a lot of problems with that approach.
00:02:11.850
Clients would call me directly, saying something wasn't working at night.
00:02:24.660
Since then, I decided that I wanted to make my life easier and focus on development and providing features.
00:02:32.200
That's why I prefer to let Heroku or any cloud platform manage those IT concerns.
00:02:39.670
So for me, the first thing to clarify is what 'cloud' really means.
00:02:45.100
We can go from real machines in our basements or data centers through VPS to fully managed environments like Heroku.
00:02:53.050
There's a huge variety to choose from, and I had a similar story.
00:03:05.050
I used to hate Heroku for their pricing, lack of features, and a lot of limits that I had to keep up with.
00:03:12.069
However, from my experience, it kind of makes sense.
00:03:17.739
For example, one of the worst features of Heroku is the read-only file system.
00:03:22.810
Setting up the initial app is a pain because you need to establish something remotely.
00:03:28.930
But when the app grows, it actually pays back.
00:03:37.420
Recently, we moved all our internal applications from DigitalOcean to Heroku for a few reasons.
00:03:44.530
Those apps started tiny; nobody was using them at first.
00:03:51.070
But it turns out we are using those apps now, and we encountered issues.
00:03:58.450
For example, there was no more memory on the server, and we needed to upgrade it.
00:04:04.150
At that point, how to do it became complicated.
00:04:09.250
This is simply because when you start, you do not prepare for high scale; you just put it there and expect it to run.
00:04:21.010
But then it's easy to overlook when you should start taking care of hosting.
00:04:28.240
Right now, we moved everything to Heroku. When the app is too slow, we can increase the resources in five minutes.
00:04:33.669
That's the main point for me.
00:04:40.990
In addition, if Heroku is too expensive for you, there is an alternative called Dokku.
00:04:46.420
It's like Heroku written in a hundred lines of Bash.
00:04:50.350
It turns out it uses Heroku buildpacks, so pretty much, what you can deploy to Heroku, you can also deploy to Dokku.
00:05:03.370
You can buy a cheap dedicated server, for example, from Hetzner.
00:05:08.530
You can have a very strong 16-core machine for fifty dollars.
00:05:13.810
After that, you can install Dokku and run multiple staging servers; the load is usually very small.
00:05:19.810
Of course, this depends on the application, but you can run twenty staging servers.
00:05:29.900
So, I would recommend giving it a try.
00:05:50.350
My comment on that would be that at some point your disk will fail, and you'll have to learn the process of replacing it or requesting its replacement.
00:06:04.510
So that's the issue with relying on dedicated hosting.
00:06:10.150
My recommendation would be to start with something like Heroku, which does almost everything for you.
00:06:15.910
If you're not comfortable with it due to pricing or lack of features, just explore what's out there.
00:06:24.760
You might want to move to something more optimized, but starting with Heroku provides a path to dedicated servers.
00:06:30.430
Starting with Heroku means less maintenance and more ease of use for developers.
00:06:37.090
Do we have any questions about Docker itself?
00:06:42.120
Doku is based on Docker, and Docker uses a layered, content-based filesystem, which can break sometimes.
00:06:50.050
You need to still upgrade your system and Ruby versions, so there's quite a bit of maintenance involved.
00:06:56.640
It works well for apps but not really for data storage.
00:07:04.540
So, you have to be aware of losing your data if you accidentally delete a container.
00:07:10.080
And that's why you should always have backups.
00:07:17.420
Okay, let's move to the next question.
00:07:18.400
What does your deployment process look like? What tools do you use?
00:07:25.680
I can start this time. I like Git-based deployment, like Heroku.
00:07:31.800
If you don't want to use Heroku, you can use deploy from repositories.
00:07:39.620
Deployment is an easy task; you just have to place the code on the server and run certain processes.
00:07:46.030
I don’t find the need to use Capistrano for that, as I already have a tool that does it for me. I like to push code to some branch and expect it to be deployed.
00:08:02.370
For simplicity, I prefer tools based on Git.
00:08:09.430
Another option is a project called Mina, an alternative to Capistrano that generates batch scripts with all the processes.
00:08:16.030
For simpler projects, it's easy to set up, even simpler than Capistrano.
00:08:22.870
However, it doesn’t support multiple servers out of the box.
00:08:31.250
If you're on a hosting platform, that platform probably provides a deployment script.
00:08:38.620
But let's talk about a different concept.
00:08:44.220
If you aren't on a platform, what's your deployment process like?
00:08:52.330
You push new code to the machine, but it might also require some supporting services.
00:09:01.020
You may need to install additional software or libraries your application relies on.
00:09:07.200
For instance, you might use Chef to spin up a Chef client on the machine, which incorporates software installation and configuration.
00:09:16.420
For me, those are separate concepts, since I rarely need to push code and install software simultaneously.
00:09:24.010
I prefer to separate installation from deployment.
00:09:29.050
Okay, that's a good point.
00:09:30.220
There is a question related to it.
00:09:36.390
Ansible, Puppet, Chef, Salt—do you have any experience?
00:09:41.580
I started with Chef, so I'm familiar with it, and I consider all these tools to be quite similar.
00:09:53.480
If you're proficient in one, there’s no compelling reason to look for another.
00:10:01.580
I know all the quirks of Chef since I have been using it for years, so I wouldn't be comfortable switching to Ansible.
00:10:15.290
I've tried Puppet but didn't love it, and I found Chef to be confusing with some of its packages.
00:10:25.000
Ansible has its issues as well, like verbosity.
00:10:34.310
But the configuration files in Ansible are easy to manage and generate, which is a plus.
00:10:46.050
Also, documentation is key with these tools; understanding the problems they solve helps.
00:10:53.450
If you are comfortable with Chef, stick to it.
00:11:03.000
Ansible is easier to learn. Sometimes simplicity leads to better outcomes.
00:11:08.110
All these tools essentially achieve the same goal.
00:11:12.010
They manage installations and process start-up.
00:11:21.350
They are particularly useful for larger setups, around fifty servers or more, or simpler applications.
00:11:29.070
In smaller setups, tools like Heroku often suffice.
00:11:37.590
My point is that all these tools serve their purpose, and your choice depends on your familiarity.
00:11:44.250
I also want to mention Nix and NixOS as part of the software configuration management tools.
00:11:56.640
I prefer Nix because of its stateless approach, fully model-driven architecture, and reproducible deployments.
00:12:04.430
However, it has its challenges, as it requires capturing configurations accurately.
00:12:12.100
Being model-driven means you deploy everything based on specifications.
00:12:20.360
You need to ensure everything is documented in the expression language.
00:12:29.399
For managing dependencies, I believe tools should address your specific context.
00:12:36.202
In many environments, one might need to share configs in microservices, which can lead to management issues.
00:12:44.210
My point here is that effectively managing configurations requires careful consideration and tools tailored to your context.
00:12:52.220
Next question: how do you handle deployment artifacts and builds?
00:12:57.710
We have a continuous deployment pipeline.
00:13:00.000
We build our assets and deploy applications using a CI server.
00:13:07.780
There’s a lot of value in separating application code from assets.
00:13:15.200
That way, application servers can be treated as stateless, focusing on their core functionality.
00:13:27.000
You can simply serve your assets from external storage like S3.
00:13:35.000
This way, scaling becomes easier since you no longer need to worry about replicating state.
00:13:41.000
For deploying large files to S3, I personally prefer to use caching strategies.
00:13:45.930
You can keep your build artifacts clean before deploying, which helps avoid complexity.
00:13:53.180
In regards to npm, it’s more like dependency management.
00:14:02.200
Tools like npm are too loose in organizing dependencies, which can lead to headaches.
00:14:09.640
We handle copies cautiously, especially for large databases.
00:14:17.740
When deploying to staging, we separate concerns like data management from application deployment.
00:14:25.640
For all these processes, the approach should be to keep some level of abstraction for limited dependency.
00:14:35.220
Another thing is to keep track of backups in case any data is lost.
00:14:49.180
In terms of backup strategies, automation works wonders.
00:14:56.080
There should be a reliable rollback in the event a deployment goes south.
00:15:05.200
That’s something every team should take into consideration.
00:15:17.940
Regular backups are crucial, so even if you do an automated deploy, you’ll have safeguard measures.
00:15:26.700
In case of issues arising, rollbacks should be decided by accountability and clear metrics.
00:15:34.860
At first glance, continuous deployment plans feel daunting, but taking small steps certainly benefits.
00:15:47.420
Next question: how do you manage configuration across environments?
00:15:58.640
It's vital to keep your configuration handled cleanly, avoiding redundant work.
00:16:09.840
My recommendation is to handle environment variables effectively by keeping them in your CI/CD process.
00:16:21.720
Tool-wise, something like dotenv can help in isolating these configurations.
00:16:29.360
Using a secret management service makes it easier to handle credentials.
00:16:41.060
In Jenkins or other CI tools, you can set up environments effectively for development and testing.
00:16:55.100
Sharing configurations boosts consistency across teams.
00:16:59.360
Ultimately, avoid committing sensitive data into not secure places.
00:17:10.070
Let’s move on to the next question.
00:17:17.300
Do you have a rollback plan in place?
00:17:24.440
Yes, we have plans that revolve around robustness.
00:17:34.330
One fallback is that we have configurations stored to deal with failures.
00:17:45.050
This involves monitoring relevant logs in case setups don't work right.
00:17:51.610
Automated rollbacks through CI complexity enable a smooth transition.
00:18:00.350
Often, rollbacks depend on the routing layers with automated cloud infrastructure.
00:18:17.134
Next question: how frequently do you deploy?
00:18:22.480
We are aiming for more frequent deployments.
00:18:31.390
Although I never worked on projects that pushed changes frequently, we can work on that.
00:18:40.570
It's ideal for workflows to deploy several times a day.
00:18:49.868
With Heroku, deployments often draw small updates, which can be completed without a hitch.
00:19:03.310
In conclusion, deploying closer to the customer needs makes all the endeavor worth it.
00:19:10.450
Yeah, we are currently deploying around five times a day.
00:19:20.330
The architecture and compact nature enable us to deploy effectively in balancing load.
00:19:30.420
Remember, deploy incrementally and maintain good testing coverage.
00:19:38.950
Next question: How do you manage database migrations?
00:19:52.080
For migrations, responsibility is shared across teams. It's crucial to keep all migration well-articulated.
00:20:06.220
In regular practice, using Rails migration tooling helps guide through smooth progress.
00:20:16.570
Use associated strategies to manage rollback with migrations.
00:20:28.480
Separation between development and production with migrations helps avoid hiccups.
00:20:35.700
Even with migrations, extensive testing bridges along the way.
00:20:43.900
That way, you encounter fewer surprises on big rollout days.
00:20:52.960
New databases configured against releases also help navigate functionalities.
00:21:00.157
Another aspect discussed earlier was backup strategies.
00:21:06.059
Essentially, state management addresses several concerns in any deployment.
00:21:15.155
So if you deploy something that fails, which is common in practice, backups give you peace of mind.
00:21:25.240
And indeed, stored databases should retain clarity.
00:21:33.460
Consider having a framework that enables you to reconfigure sets of recipes.
00:21:44.780
At times, situations feel overwhelming, but seek improvement diligently.
00:21:57.680
For evolving workloads, restore schemas come in handy.
00:22:08.890
Next question: how do you handle sensitive or production data?
00:22:14.790
Anonymizing or obfuscating production data ensures sensitive details are safe during testing.
00:22:25.670
Some use scripts to handle information like emails or names, and that way the data remains secure.
00:22:36.470
Your operational environment should always reflect those best practices.
00:22:45.370
And you should definitely avoid leaking sensitive information.
00:22:55.920
So, let’s keep the focus on maintaining production integrity.
00:23:06.360
Let’s address the next question.
00:23:10.130
How do you share configuration across your Rails applications?
00:23:17.840
The difficulty comes with having multiple environments.
00:23:27.470
Standardizing configurations results deliver improved efficiency.
00:23:36.370
My recommendation would lean towards setting the configurations clearly.
00:23:45.590
And separating services allows you to innovate with cleaner builds.
00:23:55.830
Lessons learned should apply to the sharing toolkits.
00:24:02.100
Ensure everyone understands how configurations spread across services.
00:24:09.470
Let’s proceed to the next inquiry.
00:24:16.320
How do you monitor your applications effectively?
00:24:25.190
Application monitoring can include health checks or active checks to validate functionality.
00:24:34.220
Next, how do you ensure that resources are allocated correctly to your app?
00:24:41.640
Monitor usage to anticipate resource difficulties before they become a blocking issue.
00:24:51.450
Ultimately, a focus on continuous improvement makes operations stronger.
00:24:57.740
So, do allocate time to things like performance reviews effectively.
00:25:08.750
Let's move on to the next question.
00:25:16.370
What about scaling your applications over time?
00:25:26.440
Focus on reliable architecture during growth phases.
00:25:35.330
And ensure redundancy is in place and appropriately configured.
00:25:41.640
For larger projects, utilize orchestration tools for better resource utilization.
00:25:49.950
Scaling can also connect directly to deployment frequencies.
00:26:01.090
In more mature organizations, these tools integrate into CI/CD pipelines well.
00:26:10.470
Having multiple servers also alleviates pressure in maintenance phases.
00:26:19.050
That’s enough buildup for deployments; just iterate and improve.
00:26:29.460
Next question: what tools assist with operational management?
00:26:38.150
Consider having a combination of server management systems and cloud monitoring.
00:26:45.680
Many of these tools tie back to domains of reliability within projects.
00:26:53.140
Lastly, let's summarize.
00:27:00.020
Thank you for engaging in this discussion today!
