Ruby Video | Ansible - Your First Step Into Server Provisioning

Ansible - Your First Step Into Server Provisioning

by Ben Turner

In the talk "Ansible - Your First Step Into Server Provisioning" presented by Ben Turner at RubyConf AU 2014, the focus is on introducing Ansible as a lightweight server provisioning tool and how it can be effectively utilized within a modern Ruby application's server infrastructure. The speaker shares his experience transitioning to Ansible in a Ruby-focused team and highlights the importance of automating server provisioning to avoid knowledge silos and dependency on a single individual's expertise.

Key Points Discussed:
- Introduction to Server Provisioning: Turner defines server provisioning and discusses its necessity in ensuring repeatable and reliable server configurations.
- Challenges with Existing Tools: While mentioning other tools like Chef and Puppet, he describes concerns over complexity and cost, positioning Ansible as a simpler alternative.
- Personal Experience: The speaker recounts a specific scenario where his team relied heavily on documentation to manage server configurations but faced difficulties during upgrades due to common pitfalls in documentation.
- Ansible's Features:
- Installation and Execution: Overview of the straightforward installation process and running basic commands using the Ansible command-line interface.
- Playbooks and Tasks: Explanation of core concepts such as playbooks, tasks, and idempotency in provisioning, emphasizing the use of YAML for configuration.
- Modules and Idempotency: Discussed how Ansible’s modules (like user and group management) ensure scripts can run multiple times without error.
- Templates and Variables: Highlighted the use of templates for dynamic configuration management and defining variables for different environments.
- Handlers: Introduction to how handlers can restart services post updates in a streamlined process.
- Best Practices: Turner recommends organizing playbooks, separating production and staging environments, and leveraging the roles directory for modular and clean code.
- Conclusion and Takeaways: He concludes that adopting Ansible transformed his team's workflow, improved server management, and reduced manual errors, while also suggesting valuable resources for attendees interested in further exploration of Ansible.

Overall, Turner’s presentation serves as a practical guide for developers and DevOps teams interested in adopting Ansible for server provisioning, showcasing its benefits through his own team's experiences.

00:00:11.280 Thank you. Sorry, yes. I'm here today to talk about Ansible, a server provisioning framework.

00:00:18.000 I'm hopefully going to briefly cover why you might want to use something like this and take you through your first step into using Ansible and getting into server provisioning. As Matt said, I am Ben Turner, Phantom Whale on Twitter.

00:00:29.720 Primarily, I’m a Ruby developer these days, working in a team with quite a strong DevOps culture. Although we are mainly developers by trade, we try to do as much ops work as we can, which is why I got into using Ansible in the last year, at Ainex, a Melbourne-based company operating in the construction and infrastructure industry.

00:00:50.360 We have smart people meetups, Friday drinks, and that sort of thing, and yes, we are hiring. So, the story so far: who in the room is currently using any server provisioning tool? If you could put your hands up. Oh, quite a few of you. And who in the room is actually using Ansible in particular? Not so many of you.

00:01:02.719 That's good; it means there might be something worthwhile to learn here. A few of you can probably heckle me when I get it wrong. I’m going to run through a not-so-hypothetical scenario which many of you might find familiar, and certainly one we encountered quite recently.

00:01:35.119 We had just one server for a small application within a larger company. Our server was handling quite a few tasks: running Ruby on Rails (as I’m at RubyConf), serving web content through Apache, and even some ElasticSearch for our backend search capabilities with custom configuration files. We had standard server components, users, roles, cron tasks, SSH keys, and plenty of other small complexities making up our production server.

00:01:58.799 The problem we faced was that all this crucial knowledge was locked in Fred’s head, the guy who had been with the team since the start. Fred knows everything. So, whenever we needed to update the server, we could just go to Fred and ask him what we needed to remember or do. However, being a developer, Fred faces three natural enemies. One is money; he might get a better offer somewhere else. The second is the number 73 bus that might take him away from us. Fortunately, we didn't experience that.

00:02:29.680 The last and most likely threat was that he might simply forget some crucial detail. We were aware of this problem, so we did what everyone should do: we documented everything. We had a wiki page, shell scripts, fragments of code, and manual steps documented. It seemed pretty good. Whenever we needed to change something on the server, we would update our wiki with detailed instructions of what we did.

00:03:01.560 However, three years later, when we had to upgrade the server and build a brand new one, we discovered the inevitable truth: documentation is often misleading. No matter how thoroughly you think you've documented things, you might miss a small step, especially when upgrading. So we tried upgrading from Ubuntu 10 to 12 and got about 80% of the way there, but ultimately, it seemed like we had to start from scratch again.

00:03:38.280 We had what Martin Fowler coined as a 'snowflake server.' It's beautiful, it works perfectly, but it's utterly unique and impossible to recreate. This situation is what brings us to provisioning tools.

00:03:51.040 Provisioning tools allow you to define executable code or configurations to rebuild your server with the same software and configuration each time, making it identical every time. Currently, there are four main tools in this space: Ansible, Puppet, Chef, and SaltStack. Our setup involved primarily Puppet to manage our Java Enterprise applications, but since we were just a small team working on a single Ruby on Rails app, it made us look toward Ansible as a simpler solution.

00:04:40.080 Across the board, these tools serve the same purpose, so let’s look at the core elements of them. There are directives that give commands to the servers, telling them to install, configure, or run tasks. Directive scripts compile a whole list of operations that will take your server from bare bones to the desired state. There are typically master and slave nodes, where the master node drives the directive scripts and the slave nodes are set up according to those configurations.

00:05:13.840 In Ansible, directives are called tasks and directive scripts are called playbooks. One of the selling points of Ansible is that your master node can be your own machine, so you don't have to create a separate Puppet master or Chef server; you can just use your own workstation. To this day, we still do this. The client node can be any remote server without needing any special client configuration.

00:05:58.600 With Ansible decided, it's worth a go, let’s get into installing it. Typically, it's just a one-liner using pip, yum, or brew install for Mac. Once installed, you’ll have two executables: 'ansible,' which is for running basic tasks, and 'ansible-playbook,' used for executing entire playbooks.

00:06:27.160 On the client node, as mentioned earlier, you only need SSH. Once you’ve installed the necessary package, you’re ready to run against any client node because you only need SSH access. There may also be a small amount of Python required, but that's mostly on the server side; you don't need to worry about it. After SSH, you'll also need to direct your master node about where all these client nodes are through something called an inventory file, which is a simple list of servers and groups.

00:07:04.879 In the example inventory file, you can see groups in square brackets, with servers belonging to multiple groups. Once that’s set up, you're ready to go! To verify connectivity, you can execute a simple ping module, which is a very basic Ansible module that checks everything is functional by pinging the servers. If properly configured, you should see a nice message saying success.

00:08:02.080 Next, you could try executing arbitrary command-line statements using the command module. You can pass commands using the minus m flag, such as rebooting all machines, which is incredibly useful. The command module is so useful it's the default in Ansible, so you don’t even need to specify it explicitly; you can write 'ansible all' and get it running.

00:08:48.000 Often, the user accounts on your servers may not match your local environment, so you can log in as different users, specify passwords, and it uses SSH keys by default for security. You can run commands using sudo and even run commands as non-root users. For instance, if you want to log in as Bruce Wayne but run some tasks as Batman, Ansible can accommodate that.

00:09:42.000 Now let's talk about modules. We previously discussed the ping module and command module; these are just basic modules to execute arbitrary code. Using the command module again, you can create new users or groups on the server. The trick with server provisioning is you want these tasks to be idempotent—you want them to run repeatedly without having to worry about errors when the user already exists. In Ansible's case, you have group modules and user modules that help maintain the correct state without errors.

00:10:24.200 The idea behind Ansible modules is they return a success signal instead of an error when you attempt to create an already existing entity. If you write commands that just call generic commands in Unix without the proper modules, errors will occur when you run these scripts multiple times. Modules inherently manage this and allow you to confirm the existence of users or configurations without erroring out.

00:11:31.520 Ansible provides a huge library of core modules, allowing you to work with cloud services like AWS, manage databases such as MySQL, perform file manipulations, and more. For example, if you need to install ElasticSearch—which doesn’t come as a default package—you can use the command with a create clause to totally avoid duplicate installations. If you try running the same command again, Ansible will skip it without generating errors.

00:12:47.480 If all else fails, you can roll your own modules in any programming language, as long as they can accept input and produce standard output. There is even a GitHub project to help create Ansible modules using Ruby. So that's a comprehensive view of Ansible, running commands from the command line and writing your Playbook. Playbooks take your bare-bones Ubuntu server and bring it to the desired state.

00:13:53.440 Playbooks are designed as configuration files, not scripts. Although it can include loops and conditionals, the focus of Playbooks is presenting the state of your server and letting modules handle the programmatic details. They utilize YAML, a widely recognized configuration language. You execute them using the 'ansible-playbook' command followed by the name of the playbook. At the beginning of each Playbook, you specify which group it applies to.

00:15:02.160 One important concept is tasks, which detail what happens in the playbook. Each task specifies a module and its arguments. Descriptions are optional but useful as they appear in the command line output during execution, keeping everything organized. A simple task would ensure the right software package is installed, similar to how you would run the command via the command line.

00:15:43.320 To avoid hardcoding configuration-specific values across different servers, you can define variables at the top of your Playbook. For example, you might want to configure heap sizes with variables to facilitate adjustments across different environments. You can also set variables on a per host or groups basis by adding group-specific variables to your inventory file.

00:16:48.720 Using templates is another handy way to handle variables in Ansible. Templates are nearly complete configuration files with variables that need to be substituted. When you run a task that requires a template, Ansible figures out which variables need to be inputted and fills them in accordingly, creating a finalized configuration file that the tasks can use.

00:17:38.440 Handlers are another useful aspect of Playbooks, often linked to services such as web servers. When running a playbook, you can define a handler that specifies actions like restarting a service after certain tasks are complete. If you have a task that updates a configuration file, you can notify a handler to restart the service once that task finishes executing, ensuring the system returns to a desired state.

00:18:30.440 Putting it all together, an example Playbook will contain the elements we've discussed: logging in, defining variables, running tasks, and applying handlers. When executed from the command line, you should receive output indicating the tasks have been performed successfully, detailing any changes made or confirming that everything is already in place without further adjustments needed.

00:19:12.280 Ultimately, your Playbook should present a clear overview of actions taken, and if run multiple times, commands should transition from indicating changes to merely confirming successful actions, as everything is already set appropriately.

00:20:05.200 One additional tool that might enrich your experience of working with Ansible is Kce—a utility for enhancing output readability. By default, Ansible's output will show basic information, but with Kce installed, you can enjoy a more engaging output presentation, including cows as a fun mascot element. It adds character to the feed, helping to make the process feel lighter and less daunting.

00:21:16.080 As we move on from that, much of the material remaining will primarily focus on refinement and enhancement of practices built in Ansible. These will describe advanced methods of using Playbooks and keep things organized, primarily through tasks, files, and templates in a structured manner.

00:22:14.240 Utilizing looping constructs allow us to streamline tasks that require multiple repeated actions, making our Playbooks simpler and cleaner. Moreover, conditionals will help us limit actions based on server types and architecting playbooks for specific deployments.

00:23:02.520 Ansible collects information about the machine during execution, allowing the obtention of system-level details, such as OS family or available processes. These facts enable smarter and more precise execution paths based on the machine's environment.

00:23:36.520 In terms of organizing your Playbook and structure, it is key to ensure readability and ease of use. Utilizing separate files for variables, tasks, and handlers will enhance modularity and facilitate easier updates in the future, adhering to best practices established within the community.

00:24:23.840 Using a roles directory is a best practice with Ansible to provide further clarity in your repository structure and reusable components when performing deployments. This significantly reduces clutter in your primary Playbook and allows configuration files to remain focused and readable while automatically pulling in tasks.

00:25:05.480 The final takeaway message is to ensure that you keep your production inventory separate from staging. Managing multiple environments becomes easier while avoiding potential blunders can be a significant time saver. From meticulous organization to leveraging these tools to streamline server provisioning, these methodologies ensure your first step into Ansible is productive.

00:26:05.120 In summary, the practices highlighted over the past several minutes provide a foundation to ensure that your server provisioning moves toward an established automation path. As far as our experience goes, adopting Ansible has transformed our workflow. Initial hurdles regarding the framework's usability have paid off through rapid implementation and a decrease in manual errors.

00:26:55.760 I can confidently say that Ansible has provided us the procedural foundation needed to better structure and manage our server environments. Despite the challenges some may raise about this choice compared to Chef or Puppet, we’ve had a successful transition while remaining committed to the benefits offered by this framework.

00:27:27.820 For those curious to dive deeper into Ansible, I recommend looking at the new resources available for understanding the framework better. The rebranded official documentation offers timely updates, while additional resources like Ansible Weekly and Ansible Galaxy can provide insights on how others are utilizing Ansible successfully.

00:28:15.040 As you begin this journey, remember that this is just the starting point. Embrace the flexibility and robustness of Ansible, as it can significantly enrich your server provisioning efforts. Thank you for your time.

00:29:11.360 I’m here to take any questions you might have.

RubyConf AU 2014