00:00:12.320
All right, hopefully this is for everybody. So, welcome to Chef Cookbook Design Patterns! I'm Joshua Timberman. Thanks for picking my talk during this time slot. I know we had a lot of choices today. I went ahead with a one-hour talk recommendation that Mike sent out, but I still have nine slides to cover, so let's get started.
00:00:18.640
So, who am I? I write cookbooks. Enough said. I also work in web operations. I've worked for IBM and have experience in e-business, handling enterprise systems with Java and XML. I managed systems at IDE and later at Stands, where I automated infrastructure using Puppet, Capistrano, and various internal tools.
00:00:24.400
This journey brought me to HJ Solutions, which provided systems administration consulting for startups, focusing heavily on Puppet, Capistrano, and Ruby on Rails startups. HJ Solutions eventually became Opscode when we published Chef. Currently, I write many cookbooks, lead the training program, conduct consulting services, and engage in evangelism by speaking at conferences.
00:00:31.520
Now, who are you? Who here is a developer? Alright, quite a few! Who’s a systems administrator? And who is a developer that also does systems administration? Who wants to eliminate their systems administration tasks? Well, Chef will help you with that!
00:00:43.920
Now, let’s take a look at business people. Hint: if you’re an independent consultant, you are indeed a business person. I’m excited to share that we now have a logo for Chef, which is pretty cool. This means we can stop using icons like the chef from the Muppets or South Park.
00:00:52.719
Let’s have another show of hands – who is using Chef? Solo? A few people. Who is using the Chef server? Awesome! We love you all, thank you. And we really appreciate everyone here. Just for fun, I will hand out hugs afterwards. If you’re not familiar with Chef, it enables infrastructure as code. It may seem hard to read since it’s displayed in dark gray here.
00:01:10.080
But this isn’t a talk on how Chef works or other details. I have a number of talks and training classes available for those questions, so feel free to check out those resources. Also, if you scream angrily on Twitter, I will probably respond to help you!
00:01:20.200
Chef provides an MVC framework, which is really neat. So, how does it do that? The node attributes contain information about the system we are configuring. The data we have is what we will use in our configurations. The configured node acts as the view and the recipes are the controller. Since Chef recipes are just Ruby, it gives us a lot of power and flexibility.
00:01:30.720
And because Ruby is a programming language, we want to do things properly and design them well. We’ll package those recipes into what we call cookbooks, which are collections of recipes and supporting code. These cookbooks contain applicable design patterns.
00:01:42.919
Now, I'm not a computer scientist, so if you are and I’m misusing the term 'design patterns,' I apologize. We can have that discussion at the hack fest, which could be pretty cool. However, these cookbooks represent best practices, and when you apply good design patterns and build them into best practices, those cookbooks can be utilized by others.
00:01:55.000
When it comes to best practices, opinions vary. There’s a lot of opinionated software that people are familiar with and love, and there are many best practices encoded into the way systems are built and managed. Thus, it's a great idea to share those opinions, allowing others to offer their feedback.
00:02:07.679
We will be discussing cookbooks. We have many repositories and branches of cookbooks here. How many of you are contributors to the cookbooks repository? A few hands. Okay, now who here has contributed a patch to the cookbook project? Anyone? All right!
00:02:19.920
The next question is: if I told you to sign a contributor license agreement, it’s because we chose the Apache License and wrote a blog post about it. The important part of this agreement is that we’re not going to re-license your contributions; you get to keep those. We just redistribute them for you.
00:02:26.960
So, contributor license agreements are beneficial. You can still have your code reach a wider audience through our community site. We really appreciate our community, and we want to provide a platform for sharing Chef code and cookbooks, enabling ratings and social networking.
00:02:33.760
This is the newly redesigned cookbook site where you can share your work. You’re not required to sign a CLA, and you don't even need to license yourself to post, but please ensure you have the rights to redistribute the content.
00:02:44.519
If you add non-redistributable code to a cookbook and someone complains about it, we will have to take it down due to DMCA requests. Therefore, make sure you can distribute the code you provide. If you’re writing cookbooks yourself, it's essential to follow good design patterns to receive favorable feedback.
00:02:56.360
Cookbooks package up configurations and have various components apart from just recipes. So, let's discuss those components.
00:03:02.720
Who here saw the blog post by Tom Preston-Werner from GitHub? He wrote about readme-driven development, emphasizing that the first thing you should do when starting a new project is to write the README. This helps clarify the code you're set to write and allows for brief documentation.
00:03:18.400
Make sure you write the manual, as the users of your cookbook will appreciate it. After documenting, you should focus on writing some recipes. I attended a large conference in Europe this past year called FOSDEM, the largest open-source convention in the world, with around 5,000 attendees.
00:03:28.119
The talks at FOSDEM were split into different dev rooms, including the Canoe Dev room, where I heard a talk on 'Canoe Parallel.' It replaces Xargs and allows you to run piping commands in parallel. During this talk, I decided to write a cookbook, not because the session was boring but because I wanted to!
00:03:44.399
The software installation process generally involves downloading, extracting, and running configuration commands like 'configure, make, make install.' In Chef, we can achieve this using a remote file resource that downloads a .tar.gz file, followed by a bash script that extracts and builds it. However, this presents a challenge.
00:03:56.079
The challenge includes the static nature of the resource definitions, which do not allow file retrieval from alternate locations. For instance, if we want to install a specific version, we need to account for possible updates that may have occurred during our jobs.
00:04:05.760
Additionally, every time Chef runs, remote resources will download the file and check the checksum, which is inefficient. To address these issues, we can utilize customizable attributes in our cookbooks and approach this design with proper planning.
00:04:15.079
This leads us to our first design pattern: using attributes. In our cookbooks, we can create a default attributes file to define defaults, such as the URL for downloading the desired software and the version we want to use.
00:04:22.879
When defining defaults, we should also explain that some systems may clear the /tmp directory upon reboot. In fact, we can use an internal Chef configuration value called 'file_cache_path'.”
00:04:38.640
This allows us to download files to a semi-temporary location, which is more reliable. This brings me to the second design pattern: exploiting Chef’s internal values. We will rewrite our recipe for the remote file resource using these attributes and the internal values.
00:04:47.439
Now our recipes can incorporate strings and hashes, which allows for more dynamic configurations. The bash script must also utilize attributes, including version attributes, during installations.
00:05:03.799
Moreover, we can utilize Chef's node platform value in attributes files to select specific configurations based on the node's platform. By doing so, we can enhance the flexibility of our cookbooks, allowing them to operate across a range of environments.
00:05:23.520
For example, through platform-specific conditionals, we can modify configurations to meet individual requirements. This versatility is critical for deployment across various infrastructures.
00:05:30.960
So now, let’s run Chef. We can download the remote file, set the file modes, and then execute the script, which builds the software. I wrote all this in about 30 minutes because I’m particularly skilled.
00:05:45.439
Next, we will look at recipes. It’s good practice to split recipes by functionality when building Chef cookbooks. Typically, you will have a cookbook per service type in your environment, promoting modularity and maintainability.
00:06:03.360
For example, later today you will hear about service-oriented architecture and how this affects the way applications are developed. In practice, for each major service, you may have separate cookbooks and recipes tailored to respective services.
00:06:20.080
To achieve good structure, each cookbook should fulfill specific functions, such as having an Apache cookbook that builds Apache and a MySQL cookbook that configures the MySQL server. With clear responsibilities, each cookbook becomes easier to maintain.
00:06:37.440
Additionally, these recipes should strive to avoid hardcoding values since attributes should dynamically encapsulate relevant data. Chef excels in expressing infrastructure data in flexible formats, promoting best practices.
00:06:53.840
One approach involves utilizing data bags – a built-in Chef server feature that creates an accessible store for arbitrary shared data. This allows you to link various resources and data in your recipes.
00:07:12.520
Another key feature of Chef is its rich search API, providing users with powerful discovery capabilities related to node attributes, data, and other information within the Chef environment.
00:07:30.000
The combination of these features enables dynamic configurations, making it possible to publish cookbooks that adjust based on the current infrastructure. By using Chef effectively, cooks can share invaluable configurations and resourceful recipes, allowing other users to connect intelligently.
00:07:45.920
Design credit four is about creating separate recipes. For instance, applying this to a Nagios example involves organizing defined recipes that concentrate on common components for monitoring servers.
00:08:03.200
The client recipes will use Chef's server search feature to identify monitoring servers and add their configurations accordingly. This enables seamless integration of new nodes, allowing for efficient monitoring setups.
00:08:23.040
As monitoring servers dynamically gather statistics, they create an environment where new instances are discoverable, significantly streamlining configuration across the infrastructure.
00:08:40.800
For example, in our example, the server recipe will include code that searches for all nodes with a designated hostname, allowing for appropriate host configuration file generation.
00:08:59.760
We can accomplish this through data lookup and configuration templating. Each time a new server joins the Chef system, it automatically comes under monitoring without any manual intervention, showcasing Chef's streamlined operation.
00:09:20.000
If you use Chef Solo, you won’t have access to many of these features. The Chef server provides persistent node data, arbitrary infrastructure data in data bags, search indices, and a robust API for creating rich environments.
00:09:38.759
Should you wish to continue using Chef Solo, you can check for it using an internal configuration value, allowing you to adapt your environment accordingly.
00:09:58.960
Hardcoding values presents an anti-pattern tendency, making it crucial to share customizable cookbooks that communicate clearly with other users.
00:10:14.760
It's vital to document default attribute values thoroughly within your recipes, as it promotes usability among users – roles play an essential part in this.
00:10:29.680
Once users define roles, you must ensure they permit attribute value overrides, enabling adaptability across shared systems.
00:10:48.160
Moving on to templates and files, these assets are crucial during configuration. For instance, you may need to distribute static configuration files uniquely tailored to different platforms.
00:11:06.759
In the case of MySQL, varying config options arise across platforms, illustrating the necessity to maintain platform-specific attributes.
00:11:17.919
Templates with embedded Ruby (ERB) work extraordinarily well with Chef for generating dynamic configuration files based on node attributes or generated values, making sharing files across systems considerably easier.
00:11:33.839
Libraries within Chef can provide a variety of benefits, including lightweight resource and provider helpers. If Chef doesn't have a specific resource for managing an application, you can create a custom solution.
00:11:54.480
By utilizing library resources within your cookbooks, you can effectively manage additional functionalities and customize code logic for complex applications.
00:12:05.920
Definitions are becoming obsolete since they often resemble resources but lack flexibility. Rather than using definitions, leverage lightweight resources for building new resource types.
00:12:24.159
These lightweight resources and providers contain both actions and attributes, lending themselves to a more organized, maintainable code.
00:12:42.960
As you design new resources, ensure there's clarity in their attributes, especially concerning databases and other crucial components. Utilize community resources to enrich your cookbooks and share openly!
00:12:58.560
Let’s not forget about the importance of cookbook metadata. These definitions play a critical role in establishing dependencies, especially when you're sharing or installing pre-existing cookbooks.
00:13:12.120
Include essential dependencies from standard cookbooks, such as the build-essential cookbook, which supplies necessary compiler tools.
00:13:28.560
I understand we’re fast approaching the end of our time, so let’s quickly go over testing cookbooks. You can treat these cookbooks similarly to other Ruby projects using different testing frameworks.
00:13:40.560
After validating your code, make sure you document the processes to share your efforts and insights with the community. Cookbook examples are available on the community site, ensuring that resources are easily accessible.
00:13:58.480
I will be at the Engine Yard Hack Fest tonight if anyone has further questions about Chef or anything we've discussed today. Thank you all for your time!
