00:00:15.360
Hey everybody, my name is Kyle Rames and I'm a Developer Advocate for Rackspace. Today, I'm going to cut through some of the fog of cloud for you.
00:00:27.680
On the agenda today, we'll have a brief five-minute "Cloud 101" to go through some basic cloud concepts, and then we'll jump into working with clouds using Ruby. You'll actually see some code in this talk.
00:00:42.239
So, what is cloud? Cloud computing involves consuming computing resources like a utility, much like the power from an electric outlet. You'll pay for compute time by the hour, and for file transfer and storage by the megabyte, similar to how you consume power. This resources feels seemingly infinite.
00:01:11.520
There are three different types of cloud computing: the first is Software as a Service (SaaS), which most people are familiar with, like Gmail and GitHub. We'll skip past this as I assume most of you know about it already, and move on to Platform as a Service (PaaS). This is where you upload your Rails or Sinatra application to a cloud provider, who manages it within a controlled environment, handling system administration and security, while helping you to develop a scalable solution.
00:02:10.000
The drawback of PaaS occurs when you want to do something outside of that environment, like running a necessary service that isn't offered by the provider or needing to use a library that your platform won't allow. This brings us to the third type of cloud: Infrastructure as a Service (IaaS), which is my favorite. IaaS provides a lot of flexibility, allowing you to combine various components like computing power, storage, and load balancers into virtually endless configurations.
00:02:41.280
Typically, the three types of cloud are represented in a pyramid diagram, with IaaS at the bottom, signifying its flexibility. As you move up the pyramid to PaaS and then SaaS, the flexibility reduces. Consequently, the time to build and deploy solutions increases — it's much easier to set up a solution like Github than to provision servers and install software.
00:03:24.480
When I mention cloud, many think of the public cloud, which offers inexpensive resources that feel nearly infinite. However, there is also a private cloud option where you purchase or rent physical machines and install cloud software, providing programmable infrastructure but potentially at a higher cost and with limited capacity. Additionally, there's an emerging third type known as hybrid cloud, which allows you to manage portions of your resources between secure and less secure environments, optimizing both security and cost.
00:04:49.680
Another aspect of cloud you might consider is whether to use a closed or open cloud. A closed cloud means you can't access the source code and any requests for features would result in a simple comment submission. In contrast, an open cloud, like OpenStack, allows you to view the source code, submit pull requests, and collaborate with the community to enhance functionalities. OpenStack is in fact celebrating its third anniversary today!
00:05:49.680
With OpenStack, you can choose to run it in a private or public environment, download a distribution to install on your own servers, or use a service like Rackspace or HP that offers open cloud solutions, enabling you to move between diverse environments seamlessly.
00:06:21.390
So, that covers a brief overview of cloud concepts. Now, let’s dive in and start manipulating clouds using Ruby. We're going to use the Fog gem, which is considered the number one cloud SDK according to rubytoolbox.com.
00:06:54.960
Fog has over 2 million downloads and boasts around 300, almost 400 contributors, making it a very active project. This project sees 30 to 60 commits weekly, and chances are, even if you're not using Fog directly, you're benefiting from it as it powers many popular gems like Paperclip and CarrierWave, as well as tools like Chef and Puppet Provisioner.
00:07:31.440
Today, I'm also excited to introduce you to RUM, a new utility from Rackspace, which Charles Lowell will discuss later in his session. Since I’m a top contributor to the Fog project, feel free to come talk to me after the presentation if you have questions regarding using Fog with Rackspace, Amazon, or HP.
00:08:11.840
To see the list of providers available in Fog, we can call `fog.providers.keys`. It will return a list of around 36 providers. Some of these are strictly DNS-based, while others like Rackspace, Amazon, HP, and OpenStack provide a full suite of services. For today, we will choose Rackspace, as I know everyone here appreciates open clouds and fanatical support.
00:08:42.559
Next, we'll look at the services offered by Rackspace. This can be done through `fog.services`, which will return a hash. The hash will contain services as keys and their corresponding providers as values. We will examine the services available under the Rackspace provider. Today, we will look at `compute_v2`, load balancers, and also DNS services.
00:09:19.760
For those observant in the audience, you might notice that Rackspace has two compute services: `compute_v2` for our next-gen servers. Unless you're operating legacy servers, you should primarily utilize the `compute_v2` service.
00:09:52.640
We'll create a sample architecture using Fog. We will provision two cloud servers, set up a load balancer, and create a DNS entry that will point to the load balancer. Let's begin with the virtual machines. We'll create a compute service using the command `fog.compute.new` and pass in a hash. The first key-value pair will specify the rackspace provider, ensuring that we get the right implementation.
00:10:22.960
Next, we will include additional key-value pairs for specific configuration parameters of that service provider. For example, we'll specify `v2` as the version to get `compute_v2`, and then provide the credentials while using the Chicago data center for our operations.
00:10:56.560
We need three pieces of information to create a server: the image, the flavor, and the server name. The image could be one of the following operating systems: Ubuntu, FreeBSD, Windows, or Arch Linux. This could even be an image you created from an existing server.
00:11:38.399
The flavor represents the server configuration, combining memory, disk space, and the number of CPUs. This is where much of the cloud server's cost originates. Be mindful that some images, such as Windows or Red Hat, require licenses.
00:12:08.160
The last piece of information is simply a name for your server. Choosing a name is notably one of the hardest tasks in computer science, second to cash invalidation and off-by-one errors. When selecting a name, it's useful to keep it disposable since you'll use the server temporarily rather than forming a deep relationship with it.
00:12:41.760
The name I’m choosing is TK421. With all this information gathered, we can proceed to create the server while providing our image, flavor, and name. The server object returned might have some attributes set to nil because the server creation will be scheduled on the Rackspace cloud but hasn't been completed yet.
00:13:25.040
To check the current status of the server, we can reload it to get the latest data using the command `tk_421.reload`. After doing so, we can observe that some attributes have been populated, showing you the current state of the server.
00:14:05.840
If we want to access specific attributes programmatically, we can use the syntax `server.attributes`, which will return a hash containing all attributes. Fog also enables some meta magic, allowing you to use `tk_421.state` to check its status. At this point, we can also check the server's progress during its creation.
00:14:59.840
Since building cloud resources involves a common pattern of status checks, Fog offers a method called `wait_for`. This method will reload the server state and check if it's ready based on certain conditions agreed upon by the provider. For our specific server, it will determine if it's in an active state, has 100% progress, and is allocated an IP address.
00:15:45.640
If the block for `wait_for` evaluates to false, it will pause for a second or two before repeating the check until the condition is met or a timeout occurs. In this instance, our server was ready in a little over three minutes, at which point we can retrieve the login credentials using `server.username` and `server.password`.
00:16:45.000
When a server is created on the Rackspace cloud, the password attribute will be populated. However, when retrieved subsequently, it will not be available. Hence, it could be helpful to either store it upon creation or change it later. Personally, I prefer using SSH key authentication for improved security and convenience.
00:17:29.440
Fog even provides a bootstrap mechanism that creates a server, waits for it to become ready, and then sets up an authorized key file while locking out password access for the root user. This approach ensures we're using safer practices in setting up our server.
00:18:18.080
Once setup is complete, Fog will generate an attributes and metadata JSON file containing server provisioning details. The syntax for bootstrap is similar to server creation but includes additional attributes for the public and private key paths.
00:18:55.840
Following the completion of the bootstrap, you can start using the server, facilitated by Fog methods like `scp` and `ssh`. For instance, I can upload a setup script using SCP to `root/setup`, then remotely execute it to set up the VM as needed.
00:19:43.840
Moreover, the Rackspace team employs this methodology for load testing the Repose project, which is our reverse proxy positioned in the cloud to handle authentication and rate limiting functionalities. We've merely begun exploring what can be achieved with cloud servers, and I urge you to delve further into the documentation.
00:20:25.760
Now let's talk about the collections within Fog. Earlier I referenced the `service.images` method, which returned a collection of images. This collection inherits from an array, meaning it allows us to utilize array-related methods, which is why we've referred to methods like `first` and `find`.
00:21:09.360
Also, there's a `Fog::Collection` containing the inner workings that aren't especially interesting. The images class, however, includes multiple methods such as `all`, which facilitates lazy loading when accessing data within the collection.
00:21:45.920
If I had known an ID for an image, say the Ubuntu one, I could have retrieved it directly with the `get` method. The class also has a `create` method, emphasizing our architectural integration with RESTful design principles. Using `create` not only creates a local copy of the VM but also the remote server.
00:22:29.040
To explore service collections, we can invoke `service.collections`, which reveals the various collection types available for our compute service. Three of these collections have been used already: servers, flavors, and images, alongside two others: attachments and networks.
00:23:05.760
Now that we've seen how the model layer operates via both compute and load balancer services, we should discuss the request layer, which provides a direct mapping from the cloud to Fog. The request layer offers the most efficient way to interact with the cloud but lacks portability across providers, which is why gems like Paperclip, CarrierWave, and others favor the model layer.
00:23:45.760
You would typically resort to using the request layer only if a specific feature was unavailable through the model abstraction. To see what's available in the request layer, you can employ `service.requests`. For instance, I can extract various requests supported by the storage service from Rackspace.
00:24:36.760
Let's consider the practical scenario of creating a DNS record for our load balancer. We begin by creating a new DNS service instance using `fog.dns.new`, specifying our provider. Similar to other services, we pass in credential information, leading to an increasingly familiar pattern in how we apply services in Fog.
00:25:35.520
For demonstration, we will employ the request layer to create a new DNS record and check the available requests through `dns_service.requests`. From there, we will utilize `list_domains` and `add_records` methods to browse existing domains and subsequently add our desired record.
00:26:30.400
When we retrieve domain details through `service.list_domains`, we can expect an X-Conn response, allowing us to easily manage domains with Ruby objects rather than raw JSON. This transformation is automatic, providing a smooth programming experience.
00:27:31.760
Programming practices are evident here, including the extraction of headers from the response, which can be important when handling various cloud services that utilize data transfer methods, especially within storage services.
00:28:20.720
Let’s proceed by adding our DNS record. We will obtain the desired load balancer's IP address programmatically, then supply the details through `service.add_records`. This will include an A-type record associated with the IP address and set a desired TTL if necessary.
00:29:13.520
To wrap up, this example serves as an illustration rather than a definitive architecture. In a real-world application, we would likely implement additional components like databases, monitoring, and secure networking between VMs.
00:29:55.680
I want to conclude by providing a deeper examination of how Fog handles requests. For instance, I recall the ID of the Ubuntu image we discussed. We can retrieve it using `service.images.get`, verifying that the collections class indeed invokes the request layer.
00:30:51.840
The model layer should never directly access the HTTP layer; maintaining a clean architecture is crucial. When digging deeper into the Get method, we observe that the request layer strategy aligns neatly with the architecture of Fog.
00:31:45.360
By employing the request details obtained from Rackspace, we can conduct proper requests with set headers and expected response parameters. The token received from authentication will remain valid for approximately 24 hours, classifying all our requests.
00:32:39.360
For testing purposes, Fog's integration allows for mock data via commands like `fog.mock!`, or the usage of the VCR gem to record request details for future tests.
00:33:06.000
And that is my presentation! Thank you for attending.
