Building Virtual Development Envrionments with Vagrant

Chef

Mike Hagedorn

1 talk

#vagrant

#development-environment

#ruby

#chef

#puppet

Building Virtual Development Envrionments with Vagrant

by Mike Hagedorn

The video titled 'Building Virtual Development Environments with Vagrant' features Mike Hagedorn, who discusses the benefits and implementation of Vagrant for creating virtual development environments. The speaker begins by metaphorically linking the high-stakes pressure of a historical rocket launch to the challenges faced in software development, emphasizing the significance of reliable infrastructure. Vagrant, a tool that allows users to create and manage virtualized environments, is introduced as a solution to common development hurdles. The discussion is structured around several key points:

Introduction to Vagrant: Hagedorn explains how Vagrant simplifies the process of setting up development environments, making it easy to replicate configurations and onboard new team members efficiently.
Advantages of Virtualization: The benefits of using Vagrant include isolation of applications from their cluttered environments, repeatability of setups, and the ability to use consistent instructions for team members, overcoming the common problem of 'it works on my machine'.
Provisioning Capabilities: Vagrant’s ability to provision environments using tools like Chef or Puppet is highlighted, allowing for effective software package management and system configuration.
Practical Demonstration: Hagedorn demonstrates a simple installation and setup using Vagrant and Ubuntu, showcasing the command-line interactions and how to manage virtual machines effectively.
Working with Rails Applications: The video emphasizes practical application through a case study involving a Rails application called Fat Free CRM. Hagedorn elaborates on installing dependencies and configuring the application for local development using Vagrant and Chef. This part illustrates how Vagrant can enhance the ease of deployment and collaborative development efforts.
Best Practices: The importance of documentation, version control, and well-structured recipes in Chef for environment consistency is stressed to maximize efficiency and reduce discrepancies between various environments.

In conclusion, the combination of Vagrant and Chef represents a powerful approach for developers, providing tools that foster repeatable and controlled development environments. The speaker encourages continuous learning and exploration with these technologies to enhance software development workflows.

00:00:00 Hello, world!

00:00:20 The sky is falling, that's what the newspapers said all over the world with remarkable consistency. Everyone was freaking out in the first week of October 1957. If you know your history, and you're a techie, you might know that this is when the Russians launched Sputnik, catching everyone in the U.S. by surprise. Normal people on the street were fearful that the Russians would invade next.

00:00:39 The team that was most intimately concerned with this, more so than the average person, was the team of engineers and scientists who worked on the Vanguard project. Vanguard was the U.S. Army's attempt to do the same thing that Sputnik was trying to do. Because of Sputnik, the U.S. was pushed into creating NASA just a month later. President Eisenhower signed the documents to bring NASA into existence, but this was before that event.

00:01:02 The team from the Vanguard project was under a great deal of pressure, as you can imagine. They had been trying for several months to reach a critical milestone and hadn't been successful yet. A month later, the Russians did it again in December, as if to say we can do this anytime we want. Therefore, there was an immense amount of pressure on the team to prepare for a launch on December 6th.

00:01:27 The team gathered, conducted all their tests, and felt a great deal of pressure; there was a lot riding on this launch. Prestige and funding were important, does that sound familiar to your projects? I don't know if you know what happened on that launch, but I have a little surprise. I need to get a microphone for this, as it's important for you to hear this.

00:02:10 Can you hear that? Great. Here we go! This is a wave of advanced publicity focused at Cape Canaveral, Florida, for the launch of test vehicle three of Project Vanguard, a preliminary test for the scheduled launching of a 21-pound satellite in March. What happened next is already unhappy history.

00:02:30 It was another setback for the United States in the race to outer space. Here are official Defense Department films of the launch of the 72-foot missile, which lost thrust and fell back to Earth in mere seconds. The exact cause is classified. Neither the satellite program nor our missile development was affected, said Acting Defense Secretary Quarles. It was only an incident in the perfection of the Vanguard satellite system.

00:02:47 Now, to contradict what the man here said, they actually did know what caused this failure. I know this because when I worked at NASA, my boss was part of the analysis team that investigated what happened during the launch, and you might be surprised at the cause, or maybe not.

00:03:06 It turns out that every time they launched a Vanguard, they had a team who would come in to clean the launch pad. They would fix things like wires that got messed up or pipes that got destroyed, as it can be a violent process. One critical component tended to get damaged was an umbilical that plugged into the base of the rocket. In this instance, the umbilical is simply a glorified term for an extension cord that connected to the rocket base. Every time they launched, the connector at the end would get damaged. The electricians would come in, do their jobs, and replace the damaged connectors, but the problem was that with every fix, they inadvertently cut the cord shorter until the day of this fateful launch.

00:04:06 On the day of the launch, while the electricians were doing their job, they introduced an unaccounted variable into the system. The point here is that we often do the same thing with our systems; everyone is doing the right thing, and nobody can fault the electrician for that. Still, it can create catastrophic situations, like what happened that day, as the launch control system was configured to handle two primary systems on that rocket: a ground-based system and a primary flight control system.

00:05:11 The ground-based system managed tasks like starting the pumps and igniting the motors. When the umbilical was pulled, it signaled to switch the computer systems from the ground-based to the primary flight control system. However, due to its premature removal, the first system shut down as it was supposed to. The secondary system, however, sensed that it was too early to start up, leading to a complete collapse of the launch.

00:05:50 Interestingly, the satellite itself was ejected from the rocket and remained intact. Currently, it is displayed at the Smithsonian. This lengthy introduction leads me to my title, 'Building Virtual Development Environments with Vagrant.' However, I would subtitle it 'Stop Cutting Cords,' as what we do also impacts our deployment processes, just like what happened with the rocket on December 6th.

00:06:03 My name is Mike Hagedorn from Houston, and this is my Twitter handle. I work at silverchairsolutions.com. Today, we will discuss how to virtualize your development environments with Vagrant.

00:06:20 First, we will cover some reasons why Vagrant is a good idea. I believe I have given you quite a compelling reason already, but we will tease out some other benefits. We'll start off with the simplest 'Hello World' installation, then move on to more complicated setups using provisioning with Vagrant, which is where it starts to get interesting.

00:06:47 Our examples will progressively become more complicated, culminating with a real Rails application so that we can see how to actually apply this for the tasks you are doing. We'll also look at some other cool functionalities.

00:07:11 Let me start by asking, how many times has someone come to you and said, 'Hey, look at my cool website! Just clone it from GitHub and run rails s, and it boots up'? That happens to me about maybe five percent of the time. The reality is much different. I often have to perform a lot of complex setup before I can even run the application.

00:07:37 It's typical to install many dependencies, and I often feel like I'm performing magic incantations. Getting everything right sometimes feels like a ritual sacrifice. It's not documented well, which can lead to confusion and unexpected issues.

00:08:04 Take a look at this idealized view of our server setup. It includes your operating system, application server, database, and everything else in a neat arrangement. However, that's more of a fantasy. In reality, your application stuff is mixed with user-space clutter: conflicting program versions and unnecessary components consuming resources. This complicates our tasks.

00:08:27 There is often no isolation between our applications and the clutter on our machines. The installations are not repeatable; we rarely remember the exact steps we took to set everything up, which can be critical. Furthermore, instructions provided to teammates can lead to inconsistent results; it works on my machine is often not a valid response, yet we all have used it.

00:09:09 A really good solution for these problems is virtualization. If you virtualize, even with all your user clutter, it's neatly sandboxed away. The core elements of your work are encapsulated within the virtual OS, which is isolated from the rest of your system. This makes your application code siloed and ensures your operations are repeatable, controlled, and reliable.

00:09:46 Of course, this is assuming that you utilize source control for your scripts and configurations, which you would likely do anyways if you are adopting this approach.

00:10:08 Another significant advantage of virtualization is saving time when onboarding new team members. Typically, you spend a significant amount of time configuring their systems, especially with complex setups like message queues. By using Vagrant, you can have your server configurations version-controlled, providing another level of ease.

00:10:38 Moreover, one of the most interesting benefits of this approach is the flexibility it provides for conducting 'what-if' scenarios in your development stack. Want to test a different message queue or database? This setup allows you to test these changes easily without affecting your test servers or other users.

00:11:04 Vagrant acts as a layer that allows you to manage all of this. It was created by Mitch Hashimoto and John Bender, with the first GitHub commit noticed in January 2010. It’s been around for a while and is currently in version 0.82. Perhaps a joke about never wanting to reach version 1.0 is in order.

00:11:29 Vagrant heavily leverages VirtualBox—a well-known virtualization solution that was originally developed by Sun Microsystems and later acquired by Oracle. It's free, works well, and is widely recognized in the industry.

00:11:55 Vagrant allows you to provision using Chef or Puppet. I won't dive into the debate on whether to use Chef or Puppet here; I've primarily worked with Chef in my examples, but there are other provisioners accessible through an extension API, including the possibility of using bash scripts if desired.

00:12:25 Another excellent feature of Vagrant is its ability to map shared folders from your host directory into the running VM. This allows you to collaborate and develop effectively, ensuring your work is reflected live in the virtual environment.

00:12:54 Installing Vagrant is quite simple; you install it like any other Ruby gem with the command 'gem install vagrant.' This setup is relatively mundane, but the interesting part comes with Vagrant boxes.

00:13:13 Vagrant boxes are similar to AMIs in the Amazon EC2 world. They are images of boxes that you can clone and work from. The 'vagrant box' command allows you to download an image from the web and store it in your home directory under a directory called 'boxes'. Each project utilizing Vagrant can use this as a fresh copy of the base box for mutation without changing the base version.

00:13:48 The distribution I mentioned happens to be Ubuntu, which is commonly used, but there are other options available at the Vagrant boxes website. After downloading the box, you create a project directory where you'll do all your work with Vagrant and run 'vagrant init' to generate a Vagrant file. This file contains all the necessary metadata to boot the machine and map various ports.

00:14:28 The generated Vagrant file is quite straightforward—it specifies the base box to use. If you don’t supply a name when initializing, it defaults to the last box used. You can, however, pass a specific name during the vagrant init command to populate this correctly. Now, let's begin a demo.

00:15:06 Excuse me while I prepare to show you this demo. Thank you for your patience while I get this set up properly.

00:15:23 Let me show you how simple this process can be. I’ve pre-provisioned these VMs to save time during the demo. As I do this, if you ever get confused with Vagrant commands, just typing 'vagrant' will give you a helpful overview of the available options.

00:15:38 I will check the status of the VMs, which is probably saved since they were suspended. The resuming process will start quickly, so let's see how fast it comes back up. This is a simple Ubuntu box, and we aren’t provisioning it for this demo, which keeps it basic.

00:16:14 Now that the VM is up, I’ll SSH into it using the 'vagrant ssh' command. Once inside, I want to show you something interesting. By default, the working directory where your Vagrant file is stored is mapped to '/vagrant' in the virtual machine.

00:16:47 Let’s navigate there to see if it matches our local directory. It should look very similar, as it reflects the contents of your working directory back to the VM.

00:17:06 Now, let’s step out of the VM and create a test file just to illustrate the functionality. I’ll add some content that reads, 'This is my hello world.' Next, I'll return to the VM and check for this test file.

00:17:31 See? The text file I created locally is now accessible within the virtual machine itself. This demonstrates how Vagrant maps your working directory to the VM's environment, creating a seamless development experience.

00:18:05 We can also explore other commands available with Vagrant. These commands allow you to manage your boxes and project configurations effectively. Doing a 'vagrant box list' shows all the downloaded boxes in your home directory under the hidden '.boxes' category.

00:18:24 Vagrant destroy is a command that not only halts a VM but completely removes the associated image from VirtualBox if you need to refresh your environment.

00:18:38 The command 'vagrant suspend' saves the current state of a VM, while 'vagrant resume' restores it from suspension. These features are quite handy as they allow you to manage your virtual environment without needing to complete a full startup.

00:19:00 We are now going to take a deeper dive into provisioning, which is where a considerable amount of your time will be spent while working with Vagrant. It’s essential to understand how provisioning works to maximize the utility of the tool.

00:19:30 This diagram illustrates how the various provisioners function in Vagrant. You have your base Vagrant system, a layer of abstraction for the provisioners, and then the individual provisioners that perform operations on the target OS, like installing packages or deploying files.

00:20:03 When I started working with Chef in conjunction with Vagrant, I found it to be a bit unintuitive initially. However, once you familiarize yourself with the basic structure—having a directory for cookbooks and another for applications—it becomes much easier.

00:20:50 Each cookbook contains various recipes, which dictate how to set up specific components of your environment. It’s crucial to ensure your recipes effectively interact with your server provisioning.

00:21:15 As you use these cookbooks, you may encounter some challenges, particularly when it comes to the syntax and structure. However, with practice and experimentation, the learning curve becomes manageable.

00:21:40 Moving forward, let’s look at provisioning for a Rails application, providing a vital context to understand real-world application development. We will go over the necessary configurations, which will equip you to setup the environment effectively on your local machine.

00:22:05 In this example, we'll set up a complex Rails app using Vagrant and Chef. Fat Free CRM is the application we’ll use—a larger Rails application that provides an excellent case study for deploying using Vagrant.

00:22:32 Below we see a basic structure of our Vagrant configuration file, which provides crucial details for the virtual environment. The process begins by initializing the Vagrant file, which we will customize to best suit our application needs.

00:22:55 After initializing the Vagrant file and editing it for our specific project requirements, we will use 'vagrant up' to start the virtual machine and begin provisioning our environment.

00:23:15 Next up, let’s examine the primary recipe list that we will be using to configure Fat Free CRM within our Vagrant setup.

00:23:28 As we look through the various recipes for setting up the application, it’s clear that we still need a reliable method for installing and running our app effectively.

00:23:51 In the process, we will ensure all dependencies are installed and proper configurations are in place for MySQL and necessary gems. Track all changes meticulously; this is important because it sets your environment up for success.

00:24:18 We will also address necessary tasks like data migration and loading sample data with rake tasks, which can help in setting up a complete development environment efficiently.

00:24:56 Overall, using Chef in conjunction with Vagrant allows you to create repeatable, controlled environments, which can greatly enhance the development process and mitigate potential risks associated with different environments.

00:25:33 Additionally, you can package your Vagrant configurations, which facilitates sharing your setup with other members of your team. This means others can execute their applications without manually setting up everything you've already configured.

00:26:10 As we progress through this implementation of a Rails application, we will focus on optimizing our settings. It’s also an opportunity to dive deeper into provisioning tools like Chef further to cement our understanding.

00:26:37 Making sure that recipes are written correctly goes a long way in ensuring the successful implementation of your applications. This becomes especially necessary if your app involves complexities like background jobs or webhooks.

00:27:08 Exploring the intricacies of Chef will certainly pay off, as it supports continuous integration principles, enhancing the overall cohesiveness of your development process.

00:27:39 The groundwork being laid here is significant not just for your immediate project but also in establishing best practices that can be applied across your organization.

00:28:10 Realizing the overall architecture of your development must align with your production setups to avoid discrepancies down the line. This can enhance the developer experience and streamline the testing processes.

00:28:42 As we venture deeper into using Chef with Vagrant, let’s focus on finalizing our recipes to ensure we are setting up the environment as closely as possible to mimic production configurations. Our configurations must encapsulate everything we require to test effectively.

00:29:16 After completing the setup, you will be able to run your Rails application locally, making necessary changes effectively without issues. This flexibility is an enormous advantage and allows you to adapt quickly to any development needs.

00:29:40 In conclusion, the combination of Vagrant and Chef offers a powerful toolset for managing virtual environments, ensuring that all team members can contribute effectively while minimizing discrepancies between environments.

00:30:04 By implementing these strategies, you'll not only enhance your current development workflow but also set a precedent for future projects, leading to greater success and efficiency in your processes.

00:30:38 Now, I'm opening the floor to any questions you may have regarding Vagrant, Chef, or how to implement these technologies into your workflow.

00:31:00 If you have any inquiries about general systems or specific scenarios you've experienced, feel free to ask. I'm here to help clarify and support your understanding.

00:31:23 Thank you for being such an engaged audience! It's been a pleasure sharing this information, and I hope you find it useful for your future endeavors.

00:31:45 Stay curious and keep developing, and I look forward to seeing the great things you all accomplish with Vagrant and Chef!

LoneStarRuby Conf 2011