Ruby
Build the perfect web application with these 12 weird tricks
Summarized using AI
Build the perfect web application with these 12 weird tricks
by Ole MichaelisIn this talk, Ole Michaelis explores the concept of the '12-Factor App,' a manifesto by Adam Wiggins which aims to outline best practices for building cloud-ready web applications. The discussion emphasizes that while the ideal of a perfect software doesn't exist, following these 12 guidelines can significantly improve application development, scalability, and maintainability.
Key points discussed include:
- Codebase Management: One codebase tracked in version control should exist with multiple deployments.
- Dependency Management: Dependencies need to be declared explicitly and isolated to avoid conflicts. The importance of managing versions through tools like Gemfile is highlighted.
- Configuration Handling: All configuration should be stored in the environment rather than in the application code, to simplify management and enhance clarity.
- Statelessness: Applications should predominantly operate as stateless processes, which involves treating backing services as attached resources.
- Separation of Concerns: Administrative processes should run separately from application functions to avoid performance bottlenecks during user requests.
- Resilient Architecture: Applications must support quick startup and graceful shutdown to maintain performance and reliability, particularly in cloud environments.
- Logging as Event Streams: Logs should be treated as event streams, utilizing standard output or standard error for error messages and debug information instead of log files.
Anecdotes from Michaelis's experience in web development are shared, illustrating challenges like dependency management in PHP and the evolution of practices in Ruby on Rails. He emphasizes the need to adapt to modern methodologies to avoid common pitfalls encountered in earlier development environments.
The conclusion reiterates the significance of adhering to these principles, not merely as a theoretical guide but as practical advice to create robust applications. Michaelis encourages the audience to read the full 12-Factor Manifesto for in-depth techniques and insights, advocating for a shift towards these best practices in software development.
00:00:03.240
Let's start. This is my second talk today, and I'm still very excited to be here. I had lunch, so I'm a bit more tired now. Who has seen my first talk? Wow, okay, almost everyone. This is cool; the relation is not too hard, so it doesn't matter if you've seen it or not.
00:00:16.560
The title of this talk was actually a test. I wanted to see if these clickbait titles work for conferences. And you know what? They do! That's why we are here. This talk is mainly about the 12-Factor App, which is a manifesto written by Adam Wiggins, co-founder of Heroku, describing the ideal cloud-ready web app. But it's more than that; it should be the standard way of writing applications because it includes tips and tricks for building your web app. For instance, logging should only go to stdout, and all configuration should be done via environment variables. It's about building robust and scalable systems.
00:01:01.519
If everyone followed these 12 simple principles, we could share tooling across programming language borders. Imagine having one pre-built logging and metrics solution, one and only way of configuring your project, and the one way to run it successfully.
00:01:32.760
I've been doing web development for almost 10 years now, which feels like forever. That's why I want to talk about history—how it all started for me. I began with PHP. Back then, I used a tool that looked something like this. Does that look familiar to anyone? Great! What I loved most about this tool was that it provided continuous deployment. Whenever I hit save, my PHP files were sent directly to the server. How great is that?
00:01:46.560
Sure, there weren't any tests involved, and if a colleague saved after me, my changes might have been lost, but this was just how we did it back in the day. If you were working with Ruby during that time, it was around the same period when the first versions of Rails were created. I’m not sure how Rails development looked back then, but I think it was chaotic. In my first company, we managed dependencies using Submodules, which felt effective at the time, but in hindsight, it was a pretty silly choice.
00:02:01.520
We called our shared code 'common,' which was stupid because nobody in the company understood it, but all the projects used it. The most important code was thus poorly maintained. Fast forward to today, and we manage dependencies differently. Now, we simply add everything into our Gemfile, run 'bundle,' and everything is magically there.
00:02:36.480
Here’s a question: why do we manage dependencies this way? Have you ever thought about it? It's not just because someone told us to. There are good reasons for that, and the first reason that comes to my mind is the explicit management of dependencies. In my PHP days, I had no idea what version of dependencies my project was using from a common library. With Ruby, we get version pinning through the Gemfile.lock, giving us a clear overview of our dependencies.
00:03:19.239
We have examples and advantages with dependency management, but we also have commands that are complex, which can feel repetitive. Typing ‘bundle exec’ over and over does not feel DRY (Don’t Repeat Yourself). What I want to express is that this explicit management allows us to isolate dependencies without leaking into the global system. Dependency isolation is important because it ensures that our dependencies are not globally available, avoiding potential issues that arise when multiple applications are running on the same server with different dependency versions.
00:04:12.519
Today, when we start new projects in Ruby, what most of us do is type 'rails new' followed by the project name. This generates a lot of boilerplate code, which can be both helpful and overwhelming. I recently generated a fresh Rails configuration, and it produced 668 lines of code! Why is there so much configuration in a project I haven’t even touched? Other frameworks like PHP have much less, like 160 lines in their Symfony framework.
00:05:04.400
This excessive boilerplate often includes configurations that make sense but can feel overwhelming. There are initializers, routes, and other configuration files, but I believe we should strive to configure our environment outside of our application's code as much as possible. Configuring the environment within the code can lead to confusion. For example, secrets and database hosts should be configured in the environment. Different developers on a team might have different database backends for their development, which complicates matters, so let's keep configurations external.
00:05:54.959
Moreover, if you switch your application to environment-based configuration, fail fast! If you attempt to read an environment variable that isn't set, your application may fail in unexpected places. By implementing a fetch method for environment variables, you can catch these issues earlier in the execution process. There's a great blog post by Grouper on how they switched their project to environment-based configuration; it's full of practical tips.
00:06:59.600
Another important point is that 'true' in the environment is actually a string. Most of you are familiar with how environment configurations work, but my experience includes a particularly challenging example involving SQLite that my colleague implemented. Using SQLite as a default database combined all the necessary state into a single location, which created problems when users had to connect to specific servers. Stateful applications can lead to complexities that are hard to maintain.
00:08:01.679
So, how do we manage state in a scalable way? The usual approach has been sharding, where you split user IDs between two databases. However, this adds a lot of complexity and can worsen your state management issues. Instead, I recommend adopting a stateless architecture based on processes. In this model, you handle the workload according to its diversity and scale based on performance needs. You can easily add web processes as traffic grows and manage other processes like workers and job schedulers independently.
00:09:04.640
Formerly, I spoke about how to handle server processes. You should be able to trash and restore your servers quickly. For this, you need resilient architecture and processes that start up quickly and shut down gracefully to ensure reliability. A fast startup means not installing all your gem dependencies during the process itself. Ideally, you want a bootable process that performs as needed. Graceful shutdown involves stopping the acceptance of new requests, letting pending ones complete, and then shutting down cleanly.
00:10:39.919
In cloud environments, implementing these principles is crucial. Your application should be resilient enough to handle transitional failures. The next point involves logging. The fundamental truth about logs: there’s no log file; everything should be treated as an event stream. In Ruby, when you search on rubygems.org for logging-related gems, you might find over 1,000 hits. This leads to an unnecessary fragmentation of our logging strategy.
00:11:56.799
Instead of relying on specific log files or configurations in your application, you should use stdout for logging. When an error occurs, send it to standard error. If you need debug information, send it to standard output. Your environment should then take care of logging aggregation. This keeps your application focused on delivering functionality without worrying about log file management.
00:12:58.360
As applications mature, we often require administrative functionalities. For example, a product manager might request CSV exports or daily report generation. The problem occurs when we run these heavy processes within the same web processes handling user requests. This could lead to performance issues, creating a bottleneck. Instead, you should treat these admin processes as one-off jobs—execute them in separate processes or queues without blocking user requests.
00:14:07.140
This separation is vital to prevent performance degradation that affects users. All of this is part of the 12-Factor App principles. The 12-Factor App, as characterized in the 12-Factor Manifesto by Adam Wiggins, describes a perfect cloud-ready application and should serve as a guide. I will summarize the 12 factors quickly, but each one is essential.
00:15:24.560
The first factor is codebase: you should have one codebase tracked in version control with many deployments. The second factor talks about dependencies: declare and isolate them explicitly. The third factor addresses configuration: store configurations in the environment. Fourth, treat backing services as attached resources to maintain statelessness. Factor five focuses on separating build, release, and run stages to streamline processes.
00:17:30.019
Factor six emphasizes that applications should execute as one or more stateless processes. Factor seven involves exporting services via port binding. Eight is about concurrency, where you maximize the number of processes you can run without worrying about the state. Factor nine is disposability—maximize startup time and enabling graceful shutdown without disrupting traffic. Factor ten is about maintaining parity across development, staging, and production environments.
00:19:32.320
Factor eleven explains treating logs as event streams, and factor twelve involves running admin processes as one-off processes. All these principles contribute to the creation of resilient applications that thrive in the cloud. Even if this overview gives you a solid grasp of the 12-Factor Manifesto, I implore you to read the full document because it offers valuable examples and insights.
00:21:31.479
It's only a few hours long, and you can find it in various formats such as PDF and web pages. My name is Ole Michaelis, and I’m with Jim. You can find me on Twitter, my blog, GitHub, and other projects. Thank you all for your attention!
00:22:50.000
Does anyone have questions? I enjoy discussions and would love to hear your thoughts, particularly on the topic of development and production parity. Feel free to approach me!
Explore all talks recorded at rubyday 2014
+1