Walking in the Clouds

Application Performance

Grant Schofield

1 talk

#cloud-computing

#infrastructure-as-code

#ruby-on-rails

#devops

#continuous-deployment

#aws

#application-performance

Walking in the Clouds

by Grant Schofield

In the video titled "Walking in the Clouds," Grant Schofield discusses cloud computing and its evolution at the LoneStarRuby Conf 2009. Schofield emphasizes the transformation of traditional computing paradigms into cloud-based models, likening the current state of cloud technology to older time-sharing systems. He identifies different types of cloud services, including Software as a Service (SaaS) and Infrastructure as a Service (IaaS), while advocating the growth of Platform as a Service (PaaS) exemplified by Engine Yard.

Key Points Discussed:
- Introduction to Cloud Computing: Schofield highlights how cloud computing is a return to old ideas through modern technologies, drawing parallels with mainframe time-sharing.
- Types of Cloud Services: He outlines various service models, including SaaS (e.g., Google Apps), IaaS (e.g., AWS), and PaaS, emphasizing Engine Yard's role.
- Engine Yard Cloud Features: Schofield describes pricing plans (Solo and Flex) and the concept of infrastructure as code for easy server deployment, allowing rapid scaling as needed.
- Instant Provisioning and Deployment: The talk emphasizes the advantages of instant provisioning of servers to meet modern development demands, showcasing tools like Capistrano and Chef for simplified deployments.
- Real-World Applications: Schofield demonstrates how to deploy a web application called Twerkle, showcasing capabilities such as GitHub integration, environment setup, and load balancing using HAProxy.
- Scaling and Management: He explains how scaling can occur both vertically and horizontally, along with features supporting various frameworks and technologies beyond Rails.
- Future Directions in Cloud Technology: Schofield reflects on the evolving role of system administrators and the growing need for cloud solutions, emphasizing Engine Yard's commitment to user-centered development and continuous adaptation to industry trends.

The main takeaway from this presentation is the understanding that while cloud computing offers significant advantages in flexibility and deployment, it may not be the solution for every use case. The importance of evaluating specific requirements and leveraging the right tools for optimal performance is critical in today's cloud environment. The video concludes with a call for ongoing conversation about how businesses can leverage cloud solutions effectively.

00:00:21.199 All right, I think I'm going to go ahead and get started since we're running a little late.

00:00:27.359 Hi everyone, my name is Grant Schofield. If you didn't guess from the big slide, I work at Engine Yard.

00:00:32.559 I've worked with Engine Yard since March after moving on as the primary developer of AD drive.com. I am a support engineer, and that role is a little different at Engine Yard than most places. Support engineering combines the ability to develop and debug Ruby applications with an understanding of systems infrastructure.

00:00:45.120 I am from Oklahoma City, and I always kind of looked down on Texas. However, I went to a gas station, found some really nice beer at 10 p.m., and discovered I could buy until midnight, which is a lot better than what we have in Oklahoma City.

00:00:57.280 You can follow me on Twitter under the handle @scoffield. I want to talk a little bit about the cloud in general first because I think the cloud has taken on this web 2.0 idea. It means a lot of different things to different people now. We have all these different things coming about, but they're really all old ideas. People used to time-share on mainframes and share these systems. We've just taken that idea, applied the internet, and used all these new tricks. Now we have supposedly this OS that's supposed to run completely on the cloud.

00:01:20.560 While we're evolving, I suppose we're also going back to those old ideas. There are a few different types of cloud computing. Software as a service is one of those, and I think that's the one that most people are familiar with, like Google Apps, CRM software on Salesforce.com, and Twitter.

00:01:26.960 From my perspective, the internet is evolving into a service model. Everything we do is online now. There’s also the idea of infrastructure as a service, provided by companies like Rackspace, Cloud, GoGrid, Sun, and Amazon Web Services. These companies provide the basic tools you need to actually do something on the cloud. That’s where we come in, as a platform as a service, similar to Google App Engine. Windows Azure and Salesforce are also getting into that space.

00:02:03.600 Now, let’s talk a bit about the Engine Yard Cloud. The Engineer Cloud is divided into two pricing plans, Solo and Flex, both with the same features. Flex is a beta product right now. The main difference between Solo and Flex is that Solo only allows you to run one instance, whereas Flex allows you to run multiple instances or a cluster that operates on top of AWS. This leverages useful AWS features like elastic block storage and S3 credentials for client access.

00:02:45.280 One of the significant ideas from our development team regarding the cloud is the concept of infrastructure as code. This means having repeatable templates that allow us to click a few buttons and spin up servers that are identical yet customizable. The Engine Yard stack is another key aspect that may set our cloud product apart since our lead developers have extensive experience with Nginx and apply that knowledge to our Cloud product.

00:03:10.040 Now, don’t get caught up in all of the cloud hype. Many people say the cloud is the future, but it’s not going to be the solution for everyone. You need to consider your requirements— if you're doing a high amount of input/output (IO), the cloud can actually be a liability. That's why companies like Facebook and Twitter don't run entirely on the cloud. The cloud will solve some problems effectively but won’t solve all of them. Sometimes, it’s essential to reassess whether the cloud is the right solution, or if adjustments to your application are necessary to leverage your existing tools.

00:04:08.799 In my past, I was a Systems Administrator in Boston, working for a company called ITA Software, where I managed thousands of Linux boxes. I appreciated the tactile aspects of a server or hardware—I could touch them, interact with them physically. However, what I disliked was the bureaucracy of filling out forms and waiting months for hardware approvals. That’s where the idea of instant provisioning comes in. In our modern age, with fast-paced development environments needing numerous servers, our product allows you to instantly provision servers as needed.

00:05:04.960 One of my favorite aspects is how easy deployment has become. When Capistrano was first introduced, we were amazed at how easy deployment turned, and it’s getting even easier with infrastructure as code. This flexibility lets us spin up a test server, run long processes like migrations, and ensure everything's correct before pushing to production, avoiding potential downtime or risks to the live site.

00:06:30.960 Unfortunately, as the industry evolves, there may be fewer needs for System Administrators, as cloud technology simplifies many processes. You no longer need to know all the intricate configurations to get a site up and running optimally. Now we’re going to shift gears and demo some practical aspects of our cloud environment. We’ll use a personal project called Twerkle to illustrate.

00:07:46.599 We have a choice of using Nginx and Passenger for our environment. Currently, I’ve created a staging environment reflecting my application configuration. I’ll be deploying it to our cloud, where the application relies on GitHub for version control. After providing the GitHub URL where my repository is hosted, we’ll configure additional application settings like the Ruby on Rails version.

00:08:46.599 As we proceed with the setup, we’ll need to integrate keys for deploying our application to facilitate the GitHub interaction. This involves checking the GitHub deploy key, ensuring it’s in place to leverage deployment capabilities effectively. Additionally, during instance setup, we can add gems and specific Unix packages, ensuring our environment is tailored to our application’s needs.

00:09:21.360 Once everything is set up, we’ll create instances. In this case, we will opt for three application servers. Since we have multiple application servers, we’ll need a separate database. After booting the configuration, we will see our three application servers coming online alongside the master database. It’s worth noting that our system includes built-in monitoring, providing insights into memory usage, I/O levels, and other critical metrics.

00:10:12.920 Implemented SSL certificates are just a few clicks away, allowing for either self-signed or uploaded certificates. Furthermore, backup tools using Amazon’s elastic block storage system enable incremental snapshots for system recovery as needed. We support Capistrano for deployments and utilize Chef for sophisticated deployment hooks that integrate seamlessly with your codebase.

00:11:02.639 Our deployment capabilities allow for straightforward one-click deployment systems. Here, we are closely monitoring the status and ensuring that our servers are booting in sync, waiting for one another as they come up. Meanwhile, I’ll access another mirror of the environment that I prepared earlier to illustrate a practical deployment scenario.

00:12:02.920 As we move on, I’ll make a minor code adjustment in the Twerkle application and push that update to GitHub. Before we deploy, let’s review the current version of the website. We use the IP address directly due to multiple versions running simultaneously. With the updated code in hand, we will initiate a deployment process.

00:13:14.880 When deploying, any changes that need to occur, such as configurations or SSL certificates, will deploy across application servers without affecting the database. We've utilized HAProxy for load balancing between servers, allowing for effective traffic management during deployments.

00:13:36.680 As our code deployment across the instances is completed, we can verify the successful update on the live site. Additionally, leveraging post-commit hooks from GitHub simplifies the deployment process further, allowing for rapid iterations without needing to access the deployment interface every time.

00:14:12.680 Another feature I find particularly useful is the cloning feature. This allows us to clone an entire environment to test migration processes or specific Rails versions before making changes in production. This feature streamlines environments, providing newfound flexibility and minimizing costs by avoiding unnecessary resource allocation.

00:15:23.200 Scaling can happen in two ways: vertically, which involves increasing resources like CPU or RAM, and horizontally, which means adding instances. The cloud infrastructure adapts well to needs that arise, particularly during traffic surges from viral content or significant mentions in tech news. Adjusting capacity is simple; you can add up to 20 instances, and scaling back down is straightforward as well.

00:16:50.800 When customizing the environment, we leverage Chef for configuration management, providing versatility in tailoring your systems. For instance, using basic Chef recipes allows you to install and configure diverse applications seamlessly. In addition to supporting common platforms like Rails, we also have setups for MongoDB, CouchDB, Redis, and other technologies.

00:18:23.999 With our system, we manage load balancing meticulously to handle failures gracefully by promoting healthy instances to preserve continuity. This reduces downtime and guarantees resiliency within the cloud infrastructure.

00:19:53.999 The service is designed not only for Rails applications but is versatile to integrate other frameworks. Users can manage various application types while simultaneously using their S3 storage for client interactions. Upcoming features will expand this utility, including support for Postgres databases and automatic failovers.

00:21:10.280 We understand the need for efficiency and are dedicated to providing an infrastructure that adapts to user needs. Our commitment is to continuously enhance our offerings, ensuring that fluctuating demands are met with responsive solutions.

00:21:37.239 As we explore the full capabilities of cloud technology, the overarching goal remains clear: facilitate rapid development and deployment, allowing businesses to thrive without the traditional hassle of backend management.

00:22:22.959 Our involvement in the tech community keeps us informed about emerging trends. It ensures our framework aligns well with industry standards. We believe in collaboration and shared knowledge, which benefits everyone in the ecosystem.

00:23:06.960 This collaborative approach brings distinct advantages, especially as we incorporate features that meet client needs. In doing so, we continue to drive the effectiveness of cloud services in supporting diverse applications.

00:25:05.240 As we look ahead, our mission remains focused on enhancing user experience, responsive support, and implementing innovative solutions that keep pace with the growing demands of our clients.

00:26:42.720 Thanks for joining this discussion about cloud technology. I appreciate your engagement and feedback; let’s keep the conversation going about how we can leverage cloud solutions effectively.

LoneStarRuby Conf 2009