Ruby Video
Mitchell Hashimoto
Working in Virtual Machines, the Vagrant Way
Mitchell Hashimoto
3 talks
#web-development
#development-tools
#devops
#containerization
#software-development
Summarized using AI

Working in Virtual Machines, the Vagrant Way

by Mitchell Hashimoto

In the talk titled "Working in Virtual Machines, the Vagrant Way" at LA RubyConf 2011, speaker Mitchell Hashimoto explores the transformative potential of virtual machines in development environments. He discusses how virtualization can enhance development practices by simulating production environments directly on local machines, ultimately aiming to make development easier and more efficient.

Key points from the presentation include:

  • Challenges with Current Development Practices: Hashimoto outlines a typical workflow in web development, highlighting issues with source code retrieval, environment setup, and server interactions. He emphasizes the common frustrations faced, such as lack of repeatability, difficulties in reversing changes, and non-isolated environments.

  • Benefits of Virtual Machines:

    • Isolation: Virtual machines offer built-in isolation, allowing developers to simulate service-oriented architectures effectively.
    • Repeatability: Using established scripts to set up development environments ensures consistency across different machines, mirroring production systems more accurately and helping catch bugs early.
    • Complex Relationship Modeling: Developers can simulate network conditions, making it easier to test applications under various scenarios.

  • Advantages for Teams: Virtual machines expedite onboarding processes for new hires by providing them ready-to-use environments, thus reducing setup time. Designers can also enjoy the independence to manage their testing environments without burdening engineers.

  • Introduction to Vagrant: Hashimoto describes Vagrant, a tool he co-created, which simplifies the setup of development environments using virtual machines. Vagrant allows developers to configure and manage virtual machines with minimal friction, using a command-line interface and version-controlled Vagrant files:

    • Vagrant File: Similar to a Makefile, it defines how the virtual machine should be set up.
    • Command-Line Interface: Vagrant enables users to create, destroy, and provision virtual machines efficiently.

  • Demonstrations of Vagrant: Hashimoto showcases Vagrant in action with various applications like Refinery CMS and an issue tracker named Track. He provides real-world examples of how Vagrant can streamline complex setups, demonstrating ease of use and effectiveness.

In conclusion, Hashimoto urges the developer community to embrace virtual machines and Vagrant to modernize their workflows. He emphasizes that by moving away from archaic practices, developers can significantly enhance their productivity and collaboration. With Vagrant being open-source and supported by a community, there’s a strong incentive for developers to explore its capabilities further.

00:00:30.039 I'm going to talk about working with virtual machines. We're going to dive into the grid today. Virtual machines are probably the coolest thing ever, and I think they're going to change the way you work. If what I present today doesn't change your working environment, then virtual machines will definitely change the way you work in the next five years. So, I'm Mitchell Hashimoto; that's the avatar I use everywhere. My handle is Mitchell H everywhere—my Twitter handle, GitHub, and my homepage. I currently live in Seattle, Washington, which is the background image here. I travel down here often because I work for SP. I want to give a quick shout out to LA Ruby, where you can find really smart people like Evan Phoenix. If you want some cool ideas or just want to hack with intelligent people, go to this event every Tuesday. I also want to acknowledge Seattle.rb, which I attend every week, depending on whether I'm in Seattle.
00:01:06.640 My presence here is sponsored by Engineer.art. So, let’s take a look at how we currently work. Although I will focus on web development, you should know that there are many people using virtual machines for various issues as well. The original problem I built Bean for was to solve web development issues. What I have observed is a three-step process you should all be familiar with. The first step is checking out your source code from somewhere. The second step is somewhat of a mystery, and the third is usually having the server up and running. By the third step, I mean you could go online and interact completely with the website, and every part of the website should work. I want to focus specifically on the second point, as it can often feel like black magic.
00:01:48.680 In an ideal scenario, you would have a README that looks like this. If I worked on a project and saw this, I'd be really happy because this almost never happens. Usually, what happens is you check out your source code, follow the steps laid out, and try to install it on your local machine. Most of the time, you check out the source code, run it, and see what stack traces are thrown, slowly cobbling everything together. Thank you, Brian G. Willers. Some of you might not be thinking about this right now, because this is how you work, and you're okay with that. But there are many problems with this method. The first problem is that it’s not repeatable, which means that if you have a laptop and a workstation, you have to manually replicate everything on both machines, which is both time-consuming and annoying.
00:03:06.400 The more significant issue is that you can't easily reverse changes. For instance, if you’re working on a major new feature that involves lots of code changes and perhaps some infrastructure changes, but simultaneously you’re asked to fix bugs on the live site, your only option is to try to patch your current setup for the old website and hope everything works. Alternatively, you may have to abandon what you’re doing and start over on another machine, which is not ideal. Another problem is that the setup is not verifiable. You can run your infrastructure and execute unit and integration tests, all of which may pass, giving you a false sense of security. With a manual process for setup, however, if configuration or version differences exist, you won’t know about them until it’s too late. This kind of oversight typically leads to discovering issues at inconvenient times, like late on a Friday night, only to find that the solution is annoyingly simple.
00:04:49.000 The third issue is that your environment is not isolated. You are running your web server and entire infrastructure on the same machine as your Twitter client, browser, and all sorts of other software. Although modern operating systems are good enough that this usually won’t be a huge issue, when working on mature websites with service-oriented architectures, it can be problematic. You could, for instance, be running multiple services that need to communicate with each other on your local machine. To illustrate this, I talked to someone at White Pages who mentioned that they have a separate server for processing human-entered addresses, which is used for querying information. If their developers attempted to run that service on their local infrastructure, it wouldn't work well due to the vastly different requirements.
00:05:37.480 Virtual machines present a solution to all these problems. They offer built-in isolation, which you’re already aware of. With this inherent isolation, you can run multiple virtual machines and effectively simulate a service-oriented architecture. Additionally, you gain repeatability: your company probably has systems for efficiently and safely setting up your servers. Why not use those same scripts to set up your development environments? This ensures that any bugs present in development will also show up in production. Furthermore, you can model complex relationships: you can simulate latency between virtual machines, create clusters, and test how your system responds under different scenarios, which is hard to achieve on a single machine.
00:06:52.160 For managers and directors of engineering, there are several compelling reasons to adopt virtual machines. Resource onboarding becomes faster. When you hire someone, they won't need to spend a day learning infrastructure. Instead, you can provide them with a virtual machine image, allowing them to get active contributions in just one day. Designers also appreciate this since it saves engineering time; typically, an engineer has to spend time setting up environments for designers on their machines. However, with virtual machines, designers can run their own environments and test any version of the website without sacrificing developer time. To illustrate this graphically, the goal is to transform how your development environment looks, perhaps by having multiple virtualized operating systems communicating with each other.
00:08:24.960 If you’re not excited about this possibility, you should definitely reconsider; it can dramatically enhance your workflow. Now, I hope I have convinced you that using virtual machines is a worthy consideration and that our current methods are somewhat archaic. Let me introduce Vagrant, which is a tool created by myself and John Bender about a year ago. Vagrant removes the manual steps involved in setting up a virtual machine, and since then, it has evolved tremendously. So, what is Vagrant? It’s a tool for managing development environments that allows you to describe your virtual machines in a version-controllable manner. It provides a command-line interface to create and destroy virtual machines, essentially functioning like a local EC2 host. It was designed to stay out of your way and seamlessly integrate into existing workflows.
00:09:56.320 The functionality of Vagrant consists of two main components. The first is the Vagrant file, which serves a purpose similar to a Makefile or Gemfile. It succinctly describes the specifications of the virtual machine, so Vagrant knows how to set it up. The second part includes the command-line tool that you use to spin up machines, provision them, and SSH into them as needed. Let’s check out a basic example of a Vagrant file. This is the minimal setup you need to get a virtual machine running. You can think of the 'box' parameter as akin to an AMI if you’re familiar with EC2, as it refers to a base image. Each time you spin up a VM, it doesn’t actually install a new operating system from scratch. Instead, it clones a hard drive and builds the virtual machine on top of that.
00:12:50.960 Here's a slightly more complex example: if your system engineer has Chef scripts, you can provision your VM using that information. It can be as simple as using a shell script if you’re unfamiliar with Puppet or Chef. You can use a shell script to bootstrap your VM, and then proceed however you wish. Each of these steps can be composed as necessary. Additionally, you can network the VM to have a static IP. This means when the VM starts up, you can directly access it via a URL without resolving to something outside, like the Department of Defense. If you’re running multiple VMs combined into a single Vagrant file for a service-oriented architecture, you can easily manage their configurations so they sit on the same network and communicate as needed. This allows you to setup a cluster of related services quickly.
00:15:34.539 The command line for Vagrant operates similarly to Git. You provide the Vagrant binary with the exact task you want it to perform. I recently open-sourced my examples, which illustrate these setups. However, I advise against running these on the internet as they pull down a significant amount of data, which may not execute properly, and you might mistakenly think it’s Vagrant's fault. Instead, these are meant to be executed at home. I’ve set everything up on my local machine, which demonstrates how Vagrant operates. I’ll show you the output from some of the commands based on a pre-configured setup.
00:16:32.680 For example, I set up Refinery CMS, a Ruby on Rails 3 content management system that requires several dependencies. Setting it up from scratch can be quite painful, so I created a Vagrant file and Puppet recipes for it. If anyone checks out the GitHub repository I mentioned earlier, they can navigate to the Refinery CMS directory and run 'vagrant up'. After about 15 minutes of provisioning, you will have Refinery CMS up and running on your local computer without needing to take any additional steps. The output from the process will show that it booted the VM and provisioned it successfully, allowing you to then access it via a designated web address.
00:18:18.960 Next, we have the example of Track, which is a Python-based issue tracker. Setting up Track can often be a nightmare. In fact, I had previously tried so hard to set it up that we reverted to using text files before Vagrant came along. By cloning the repository for Track and running 'vagrant up', you get a fresh installation without needing pre-configured images. Using Vagrant, all of these applications can be demonstrated as examples of what you could achieve. Imagine this applied to your own website, which can also become a hassle to set up and maintain. You can streamline development processes with similar vagrant configurations. This system allows multiple VMs to run alongside each other smoothly.
00:21:12.440 The third example I have is a PHP virtual host, which is significantly simpler. Each individual VM consumes only 512 MB of RAM, debunking the myth that virtual machines are heavy. Depending on the specifications of your local machine, you can run several VMs without even noticing a drop in performance. In fact, I frequently operate multiple VMs with as little as 4 GB of RAM, and it works perfectly fine. A notable point I want to illustrate is how Vagrant integrates into your workflow. The beauty of it is that you won't need to change how you work with tools you're already accustomed to.
00:23:04.080 You can edit the PHP files directly on your local machine, and those changes are reflected live in the VMs. For example, as I'm editing an index.php file within the Vagrant directory, you will see the updates appear almost instantaneously within the running application hosted by the VM. This quick integration demonstrates a modified workflow that streamlines the development process. While working on your own computer is powerful, sometimes you may need to access specific server resources, like log files or certain software not available in the shared folder. Vagrant allows you to simply run a command to SSH into the VM directly, providing real-time access to the machine where everything is running.
00:25:47.960 All the files are accessible in your shared folder, and you can view and edit them as needed. This method even extends to using NFS for real projects to bypass limitations set by VirtualBox's shared folders. NFS is essential for managing projects with large volumes of files efficiently. As I conclude, it's essential to understand that Vagrant is completely open-source, with plenty of documentation available. But, the community could improve its depth. If you want to try Vagrant for yourself, I have a flash drive with the base image that I can share with anyone interested. Finally, I want to reiterate that Vagrant is relevant to developers across many platforms, so make sure to tell your friends about it and give it a try! Are there any questions?
Edit
Explore all talks recorded at LA RubyConf 2011
+5