00:00:00.800
All right, hi everyone! Wow, it feels like we're at a Java conference with that startup time. Cool! Welcome, I'm here to talk about Otto.
00:00:07.770
How many people have heard of Otto? Oh, that's cool! That's more than one person. It's always hard following Ben; I followed Ben when he introduced me a couple of weeks ago at DEFCON Tel Aviv. I don't have a really cool English accent, so I'm just kind of boring. I had to start with a joke. If you've ever seen the movie 'Airplane!', this is kind of an inside joke because the autopilot is named Otto.
00:00:20.760
It actually has no correlation to why we named Otto Otto, but it was a funny picture. A little bit about me: my name is Seth Vargo, and I'm the Director of Evangelism at HashiCorp. I'm also a recovering Ruby developer. You can find me on the internet; that's my GitHub, my Twitter, and my Facebook. How many people have heard of HashiCorp before? Cool! If you haven't heard of us, we make a number of popular open-source tools. We're an open-source company, so we have Vagrant, Packer, Consul, Terraform, Vault, Nomad, Otto, and Atlas.
00:01:55.170
If you haven't heard of Otto, this next section might not make a ton of sense, but I'm going to do it anyway. So, when Otto first came out, it was announced as the successor to Vagrant. We announced Otto as part of HashiConf 2015. It was on the front page of Hacker News; it was actually more popular than water on Mars, which was kind of cool!
00:02:04.880
A whole bunch of people were upset, saying, 'Wait, but I love Vagrant, and I don’t want to change!' Actually, Otto is not the successor to Vagrant. What Otto does is wrap all of our open-source tools in a predefined and codified workflow. Let me give you the brief 30-second overview of these open-source tools. We have Vagrant, which is probably our most popular tool and is used for managing virtual development environments. It works with VMware, VirtualBox, and Parallels.
00:02:50.790
Then we have Packer, which is a tool for building automated machine images; there's our application scheduler, like Kubernetes or Mesos; Terraform, which is our infrastructure provisioner for infrastructure as code; Consul for runtime service discovery and distributed key-value store; Vault as a distributed secret store. Otto is just a prescribed workflow around all of these tools.
00:03:11.840
So when you ask Otto to give you a development environment, what it is actually doing under the hood is making calls to Vagrant to bring up that development environment for you. The difference is that, in Vagrant, you have to tell it, 'I want this version of Ruby installed, I want this user installed, I want this shared folder mounted at this path, I want Postgres installed, and I want it available on this port.' Otto makes all of those decisions for you.
00:03:33.450
For this reason, Otto is more than just a tool; it's a culture change. Because those same decisions that Otto makes for you in development are also made during the infrastructure provisioning phase and the deployment phase. If you're familiar with Heroku, it doesn't give you a ton of control over how you deploy your application. You simply push to Heroku, and they build you a dyno. You can kind of pick how big it is, but they run it somewhere on their infrastructure and choose the availability zones.
00:04:07.360
For many early-stage startups, Heroku makes a lot of sense because you don't have the funds or the need to hire an operations team. You might not be an expert in cloud providers or a bare-metal data center, so it doesn't make sense for you to manage all of those resources on your own. The problem is that, when you reach a scale where Heroku's ability starts to break down, it does make sense to scale back and start managing that infrastructure on your own to cut costs in half.
00:04:44.390
That transition has typically been very difficult. However, because we're using all of those tools under the hood, with Otto, it's very easy to take a step back and start tuning all of those knobs. The analogy I like to use with Otto compared to a different real-world scenario is like an automobile. Think about the average user for an automobile; their primary interface is the key in the ignition, the radio, and climate controls.
00:05:12.210
Now, an experienced auto mechanic's primary interface is under the hood, right there in the engine. They change the windshield wiper fluid, change the oil, and tune the pistons. That's for the advanced user, but for the majority of users, they just want to listen to music on their CD player or AM/FM radio. Otto is kind of satisfying that use case.
00:05:33.310
So, Otto is broken down into five different components, but the two important ones are Otto Dev, which sets up your development environment, and Otto Deploy, which deploys your application. We'll talk about the different phases to get from A to B, but first, I think it's important to take a step back and understand why we built Otto.
00:06:02.590
Developers really want to focus on building scalable and secure applications, but it's overwhelming and incredibly time-consuming to learn all of the tools and the rate at which they evolve in our industry. You have to develop locally, provision infrastructure, deploy, and then secure everything. Otto abstracts all of that away, but it's designed in a way that's not static.
00:06:25.430
Just because you deploy today doesn't mean your deployment will stay the same in the future. Otto is actually codifying all of that knowledge, and as new technologies come out and become available, Otto builds in migrations to bring you up to the newest version of software and the latest standards.
00:06:47.140
You don’t have to keep up with all the complex things that are changing in our industry. We rely on industry experts—who own portions of Otto, like the pillars of Otto—to dictate what the current best practices are. You adopt them and you don’t have to fully understand them to gain those benefits.
00:07:11.260
The first component of Otto is automated development environments. This is a problem that we solved a long time ago with Vagrant. You can simply run 'vagrant up'. We wanted that same development experience in Otto—it’s a single command you run right out of the box. It doesn’t matter if you’re a developer, designer, or an operations person; you get that same workflow every single time. For Otto, that command is 'otto dev'.
00:07:37.930
It's actually shorter than Vagrant. The difference with Otto Dev is that it is, in many ways, the simplest form of artificial intelligence within an infrastructure and development provisioning tool. Otto intelligently looks at your code base and, based on a few key factors, makes decisions on what your development environment should look like, as well as your production environment.
00:08:01.260
For example, Otto detects that we have a Ruby app. However, it can also detect Python apps by observing several metrics and decision-making factors, such as whether you have a Gemfile. This means you might have a Ruby app, but there are a number of considerations that play into that to determine if you actually have a Ruby app or if maybe you just have a Middleman app that’s not actually running Ruby.
00:08:29.250
This allows developers to focus on application development. My first job was for a T-shirt company, and I worked on the tech side. When I walked into the office on my first day, they handed me a laptop and a wiki page with instructions to install this version of Ruby and this version of Ruby. Then there was this complicated series of steps to get ImageMagick to compile with this gem. It was a disaster! It was never codified and almost always wrong.
00:08:55.640
It wasted two days just setting up my laptop. If something went wrong, I had to erase everything and start over again. With Vagrant, we put all of that on a VM. With Otto, we make those decisions for you so you don't have to think about all those dependencies. We solve all of the hard problems in a codified manner, so you don't have to worry about it again.
00:09:29.810
After you run 'otto dev', Otto will bring up a Vagrant environment. You can SSH right into that Vagrant environment, and you're immediately placed in the machine that is already provisioned and has the necessary versions of Python, Ruby, Node.js, or any other technology already installed. Otto's audience is primarily developers, but Otto does much more than just that.
00:09:55.290
The deployment process is particularly interesting. If I were to Google right now, 'How do I deploy a Rails application?' I'd come across a few articles. One of them is from our friends over at DigitalOcean, and it outlines many steps, such as installing Passenger, adding APT keys, and using Nano to edit a list file to add a custom Debian repo. We were really just copying and pasting random code from the internet into a terminal or script, and that wasn't considered a best practice 15 years ago, so why are we still doing it today?
00:10:22.790
If you ask anyone in the infrastructure space what modern infrastructure looks like, it resembles a setup with both private and public subnets. Behind your private subnet, you would have your actual applications, microservices, persistent state stores, web applications, or CLI applications. Over the internal network, there would be public accessibility; you might have a bastion host allowing SSH access into the private subnet, along with some type of load balancer to distribute traffic.
00:10:49.020
The problem? There’s a ton of complexity in building this type of infrastructure. Anyone in this field for a long time knows it's tough to get right and hard to do in a reproducible manner. We've built tools like Terraform, Packer, and Nomad to automate this for you, but wiring them together still presents its complexities. Terraform provision infrastructure for you, but you still have to tell it what to provision.
00:11:27.260
On the other hand, Otto brings all of that together in a cohesive manner. So why is this still the top result when you search for how to deploy applications? It's for two reasons: firstly, without Otto, it's still easier to do things as they currently are and secondly, we haven't yet reached a point in our industry where we are willing to give up this much control over our infrastructure.
00:12:02.970
Traditionally and culturally, operators have had a lot of control over their infrastructure because when things go wrong, they are the first line of defense; they’re a cost center. Every time we have downtime, it can be blamed on them, and until we break that cultural barrier—where we stop caring about the infrastructure so much with the mentality of 'pets vs. cattle'—we won't be able to cross that threshold.
00:12:33.890
Hopefully, Otto can help in that process by codifying this approach to infrastructure management. The last thing concerning Otto is microservices; they are a hot topic! If you're not familiar, microservices look like this: you have your Rails app (which can be seen as a monolith on the left-hand side) and you decompose it into smaller, independent services. Every model in your Rails app becomes its own service.
00:12:55.900
For instance, instead of having just a user model, you have a User Account Service. The advantage of microservices is that you can have separate code bases and individuals who specialize in different parts of your organization. You don’t have to release the entire app all at once; you can release iteratively. However, the downside to microservices is they add a level of complexity that increases with each service.
00:13:25.030
Each additional microservice introduces overhead and new questions like, 'How do these 50 microservices communicate with the same database? How do I discover services? How do I manage runtime configuration?' With Otto, we've sought to address that fundamental problem. Otto has this idea of an app file with a declarative syntax specifying your infrastructure.
00:13:58.800
Unlike the Vagrantfile, which is interpreted with Ruby, an app file is declarative. You specify your dependencies, and Otto automatically sets them up for you under the hood. It uses Consul for service discovery, so if you want to know where the database is, you just ask Consul and get the result. I decided to do a live demo for the rest of this, but if it doesn't work, I have a bunch of slides that show you what the live demo is supposed to do.
00:14:28.770
I like live demos! What I did beforehand was bring up this development environment by running 'otto dev', because you've all seen the matrix and it’s not very exciting to watch code scroll across a screen. You’ll notice at the top there are two things verifying that the created layer. This is something we don’t see in Vagrant. Under the hood, Otto uses layers, which are based on the virtualization layer.
00:15:03.640
In VirtualBox, these are snapshots used to build different components and save them in cache. This means that when we bring up our development environment, we aren't always installing Ruby from source; we cache those layers so that, much like Docker, we inject them into the VM. You are getting that same Docker-like workflow with layers, but it’s occurring at the virtualization layer.
00:15:35.930
So it says we are going to create the virtual development environment. Now if we scroll down a bit, you might notice a lot of shell provisioners happening, but if we look at this app, which we will do in a second, I haven’t defined any shell provisioners. In fact, I didn’t even give Otto a nap file. Then, down here, it installs Docker and sets up a Postgres container while it bundles our application.
00:16:02.660
This is a tiny Sinatra app, and it provides us with an IP address at the end. So let's take a step back and look at what this application is. I'm going to embarrass myself by not knowing how to use Vim in front of people. This is a tiny Sinatra app, and the Gemfile is quite straightforward; we have the PG gem because we will connect to Postgres, Shotgun (because I don't want to restart the server every time I change something), and Sinatra, and then Sinatra Active Record because I heard one of the maintainers for Active Record is in the room.
00:16:31.270
So if you look at all these files, you’ll see that nothing specifies what Otto should do. Otto looked at the Gemfile and the number of extensions that ended in .rb or other known Ruby extensions, determining that we have a Ruby app. At the time, it assumed the most recent version of Ruby was 2.0, which is no longer true, but that was the case when I set this up.
00:16:56.730
It cached that internally, so we are not constantly installing a new Ruby each time a new version comes out. Otto also detected the PG gem in the Gemfile, indicating that you want to communicate with Postgres, so Otto sets up a local Postgres instance. Right now, it sets up a Docker container and pulls the Postgres image, configuring everything for you automatically.
00:17:22.500
You don't need to worry about installing Postgres, configuring it, or setting up the Postgres user; Otto does all that for you by simply having the PG gem in your Gemfile! If you add the MySQL gem, it will pull down a MySQL container. Adding the Redis gem does the same thing. You don't have to declare any of these dependencies; many of them are implicit.
00:17:48.300
Next, let's look at this Sinatra app. There are some commented-out sections in here, but two functions are defined: a 'get' function to prove the app is running, and an 'echo' function to verify it is not faking it. I will now comment this out, as I didn’t do that before, and we’ll actually SSH into our instance.
00:18:14.060
To do this, I will run 'otto dev ssh', and this will drop us right in. If you've ever run 'vagrant ssh', you know it’s not really fast, and this is because Otto caches the address so it can perform super-fast operations. We are now inside our Vagrant box; it runs Ubuntu, and the shared folder is mounted—this same folder on my desktop, named 'demo', is what is mounted.
00:18:42.850
So what I want to do is run this with Shotgun, and I will explain why I have to specify the host parameter in a moment. We will spin up 'web brick', which is the most performant Ruby web server out there—just kidding for those who didn’t get the joke. The reason I have to specify the host parameter will become clearer in a moment.
00:19:07.740
If I go back to my local machine, how do I get access to this device? In Vagrant, we have to deal with a lot of port forwarding, and it's really complex—one of the bigger pain points with Vagrant. We decided that wasn't a good idea with Otto; instead, we will use a private IP space, and every Otto development environment will get its own IP while exposing all ports to the local laptop.
00:19:36.100
This way, we do not have to worry about port forwarding at all. To obtain that machine address, you simply query Otto for the dev address, and you will get an IP address (this is locally scoped). Now let's go ahead and curl that IP. I tend to get typing syndrome in front of people.
00:19:59.840
This will return a failure because Shotgun runs on port 9393. We should get output indicating that the app is running. This is pretty exciting because my live demo is proving to work, which is exciting for me. I don't know if it's exciting for you, but we just received a real request—this would be hard to fake.
00:20:25.260
Next, let’s try the echo function. If we echo 'hello', it says 'hello', and we can try injecting evil script tags, but they return as HTML escaped because I care about security in our fictitious web app. That's the development experience, but that’s somewhat boring; I made a stateless Sinatra app, and anyone could do that—I could do that on my local machine.
00:20:52.590
Now I want this Sinatra app to communicate with something stateful, like a Postgres server. I hinted that was our goal by showing the PG gem. Let's examine what that looks like. I'm going to stop this portion for now and take another look at our app.rb. I have this block essentially because I can’t type in front of people.
00:21:19.810
We configure the connection here, but one question you might ask is, 'How do I connect to Postgres?' In a microservices architecture, your Postgres instance isn't necessarily running on the same device; it's possible that it is using Amazon's RDS service or some distributed database provider.
00:21:45.960
What Otto does is set up our distributed service discovery tool, which is also open-source. To find the Postgres server, all you need to do is ask Consul through its DNS interface. I am not specifying an IP address or localhost; I am simply stating, 'Hey, where is the Postgres server?'
00:22:13.050
If I jump back over to our VM, I can run an nslookup for the Postgres service. You can see that I get a response back, which demonstrates that the application does not need to understand service discovery or how to connect to Postgres; we just tell it that Postgres is always going to live at 'postgres.service.consul'.
00:22:36.070
If we have a distributed Postgres cluster and we are load balancing across all instances, Consul would automatically handle that without needing to change anything in the application. The rest includes setting up the database and the Postgres user while creating an Active Record relation for a Post.
00:23:02.470
There are two new functions at the bottom of our app file—one lists all the posts, while the other allows me to create a post. I have a script here that will set everything up for me. Otto isn't smart enough to automatically migrate a database for you yet, so let me just run this script.
00:23:27.330
This script, which installs the Postgres client, creates the database, and creates the role with default login privileges. Now, theoretically, our database is set up, so we can run this Shotgun command again. And it worked—great! This means we are likely connected to the database.
00:23:55.720
Let's make sure our previous curl command still works. The app is running again, which is fantastic! Now, let's take a look at all the posts. Oh, we got an error. What we forgot to do was migrate the database. Great! We need to migrate the database first.
00:24:22.900
Now that we have migrated the database, the table exists and we see there are no posts. We can create a post by simply doing this. If we say the title equals 'Lou', that creates the post. Now, if we curl the get again and format it nicely, you can see it is successfully inserted into the database.
00:24:55.570
So, what we have shown is a development experience we set up without needing to do much on our part. We had to create the database, but the database service was already installed; we didn’t have to figure out the IP address or port number; all that was taken care of by Otto.
00:25:22.190
This is Otto's development experience. However, this experience is only half the battle. Now let's discuss deploying this application.
00:25:42.510
I'm going to load my AWS credentials into the environment, and then I can say 'otto infra'. What Otto is going to do now is use Terraform to spin up the boilerplate Amazon VPC for me with my SSH key and a bunch of other setups already taken care of.
00:26:14.790
This is a quick operation, and once complete, we will build a distributable artifact. At HashiCorp, we preach not deploying code but deploying artifacts; whether that's a Docker container or an Amazon machine image. This is emphasized when you deploy to Heroku: you don't directly deploy code; it compiles a slug; that slug is what's deployed.
00:26:36.950
The properties obtained here include the ability to quickly roll back, the ability to version all that, and the ability to track those changes through that versioning. Now that we have spun up an instance, which is the most time-consuming part, it's installing Consul for us to use—automatically.
00:27:09.540
Even though we didn't tell Otto that we need service discovery or that we need a Postgres database, Otto figures that out automatically. However, I am aware that the build process takes about 15 minutes, and that isn't the best use of everyone’s time, so I will jump back into my slides while we let this process run.
00:27:32.780
The build process is done via Auto Build, and the first time you run it, you might encounter an issue since Otto Build is based on version control. You actually need to commit your files to source control to allow Otto to know what to build. If you don't commit a file in source control, it won’t be included in the artifact.
00:28:00.300
Your source control becomes your single source of truth. After that, we can run 'otto build' again, and in this case, Otto is going to create an Amazon machine image. We could have configured Otto to generate a Docker image or a Digital Ocean droplet, but the default flavor out-of-the-box is Amazon.
00:28:29.330
This process takes a long time, but in the end, it will output an AMI for us. The result of this AMI is an AMI ID which we don’t even need to worry about because Otto is going to cache that locally. If you ask Otto about where the application is in the deployment process, it will tell you.
00:29:03.490
It will inform you that this is a Ruby application that is running on simple AWS infrastructure and that we have created our development environment and our infrastructure boilerplate is ready for deployment. But we have yet to actually deploy the application.
00:29:38.560
We initiated creating our Amazon machine image and the Docker container, but we haven’t executed Docker run yet or booted that Amazon AMI. The last step is the actual deployment process, which is simple: deployment takes your application code and that compiled artifact and launches it. The definition of launching varies by provider.
00:30:19.930
If you are dealing with a Digital Ocean droplet, it will launch in Digital Ocean; if it’s an Amazon machine image, it launches in Amazon; if it’s a Docker container, it will run 'docker run'. You might also use a scheduler like Kubernetes, Nomad, or Mesos to manage it further.
00:30:42.720
Once the application is deployed—this is very quick because it only involves booting up an instance—the output will provide an EC2 URL. You can execute the same commands on the remote machine that you ran locally; the only difference is you’re inside a VPC, going through a load balancer.
00:31:10.620
All of this is behind a private subnet with a single entry point. We’ve also set up best-practice infrastructure, but we didn’t have to think about it. Our database no longer runs in a Docker container on localhost; it now runs on a dedicated instance for Postgres.
00:31:35.180
Consul runs on its dedicated instance, and everything, including Ruby, will be installed for us without any manual intervention. Thus, the same development workflow with Vagrant can now be extended to infrastructure provisioning.
00:32:06.020
Finally, what Otto gives you is the ability to codify all of this. The organizational advantages of that are significant. In our research, many organizations incur multi-thousand dollar cloud bills due to non-ephemeral staging or QA environments they leave running because it takes too long to rebuild them.
00:32:35.240
With Otto or Terraform in general, it's easy to create these ephemeral environments. If you want an environment to do some testing, you can just run 'otto infra' and 'otto deploy' to set it up. This gives you an ephemeral environment that matches production and development setups.
00:33:04.050
If you want a persistent staging environment, you run 'otto infra' and 'otto deploy', and then you don’t run destroy—you leave it there. If you're done, you can tear it down and save a ton of money.
00:33:40.240
One question I often get is, 'My application isn't simple; I have 50 microservices, and it’s a web of complexity.' The app file automatically detects everything, but sometimes you might have more insight than the computer does. You can tell it about certain dependencies; for instance, here’s the generated app file for our application that Otto uses under the hood.
00:34:12.930
It detected that we are using Postgres, so it pulls in the example Postgres dependency. If you have your own Postgres requirements, you can write your own app file specifying how to install Postgres, its version, and tweaks you want to make.
00:34:29.800
You can also dictate the order of dependencies and build a giant resource graph. Some of you might be wondering, 'How do I change the Ruby version?' For those still using Ruby 1.8, you can specify Ruby 1.8 to be in there.
00:34:49.349
Otto provides the latest and greatest shiny new tools, but if you're not able to use them, you can simply specify those customizations to use whatever version you're most familiar with. After you edit, modify, or create the app file, Otto needs to compile. It's not the same as application compile—this generates a lot of files in the '.otto' directory specifying how to operate.
00:35:20.400
The compile step generates a new Vagrant file with scripts, along with a new Terraform configuration. Opening the compiled directory, you'll notice there's a lot in there, including various phases. Here’s the script Otto uses to build Ruby, which is quite straightforward.
00:35:52.560
What this means is Otto generates all of this so you have good visibility and insight into what's going on. Here is the Vagrant file Otto uses to build these images. When we ran 'otto dev', Otto ran Vagrant under the hood, using an auto-generated Vagrant file based on how it introspects your application.
00:36:25.720
So, again, we don’t have to know how to install Ruby or Postgres; Otto just does it all for us. That’s Otto! This was a brief 35-minute overview of the tool.
00:36:36.690
My name is Seth, and I've worked on all of these tools, so if you have questions about anything, including HashiCorp or Otto, I'm happy to answer them.
00:36:46.650
That’s it! That's all I got.
00:36:53.900
Are there any questions? Yes, the question is: If I have many microservices and want to deploy one or two separately, how do I do that? Otto is structured as a giant resource graph. Imagine I have two dependencies, but I want to deploy them separately; that depends on how you set up your services.
00:37:47.420
Your Postgres app file will live in its own folder in its own repo.
00:37:51.650
You can just CD into that folder and run 'otto deploy'.
00:38:00.100
If you have a top-level app that depends on others, Otto will deploy the necessary services. It analyzes what has or hasn't been deployed yet to determine if it needs to deploy additional services.
00:38:28.450
Yes, the question is: If you want bits and pieces, how do you just use the bits and pieces? That's a two-part answer. You can still push to Heroku while using 'otto dev' for local development, the same way you would with Vagrant.
00:38:39.030
Additionally, Otto is pluggable; it has a plug-in architecture, and it's written in Go, so if you want to create a plug-in to connect with Heroku, that can be done. I don’t know exactly what that process would involve, but it’s completely possible.
00:39:02.440
So, the question is, if I need multiple development environments, is that a problem? No! Each time you run 'otto dev', it will build as many machines as you specify in your app file. If you switch to a different project folder and run 'otto dev', it'll yield a different VM under the hood.
00:39:24.900
For separate instances within a single project, I think it is possible; I’d just have to double-check. The default supports running everything in Docker, but I think you can tell it to run in a different VM.
00:39:46.000
Are there any other questions? Thank you very much!
