“Infrastructure as Code” is a tenet of the DevOps community. It might even be called revolutionary if you can remember the days when virtual machines were a novel thing and physical hardware was the norm. But treating Infrastructure as Code is a tall order. Development practices have also evolved rapidly and nowadays that means continuous integration (even delivery!), automated tests, code coverage and more. How far can we go with the “Infrastructure as Code” metaphor? Pretty far, actually. We’ll use Chef, a well-known IT automation tool, to illustrate the state of the art.

The remainder of this article will take you on a journey from a blank slate to a fully tested cookbook. We’ll start by going through a quick overview of Chef’s main concepts. Afterwards, we’ll discuss a modern cookbook development process. Our sample cookbook will be:

1. Statically validated with RuboCop and Foodcritic;
2. Unit tested with ChefSpec;
3. Integration tested with Test Kitchen and ServerSpec.

Chef for Beginners

If you already know the basics of Chef and are in a hurry, you can just jump to the "Cookbook Development Process" section.

Chef uses an internal DSL (domain specific language) in Ruby. This decision has powerful implications. An embedded DSL means you get all the power of a real programming language: powerful abstractions that let you do (virtually) whatever you need; a standard library; thousands of open source packages; a strong body of knowledge and conventions; a large community. On the other hand, all that power brings complexity, which might be warranted or not depending on your use cases.

Chef has a somewhat steep learning curve, with lots of concepts and tools that have to be learned. If you’re new to Chef, I’ll introduce you to some of its concepts as they appear in the article. I assume you’re working in a Linux environment, but we’ll also cover Windows.

Earlier this year, Chef released the Chef Development Kit (Chef DK), which greatly simplifies your development environment setup. We’ll start from there so, if you want to follow along, download Chef DK now. Chef DK includes:

• chef, a command-line tool, still on its early stages, that aims to streamline the Chef development workflow;
• Berkshelf, a cookbook dependency manager;
• Foodcritic, a cookbook linting tool;
• ChefSpec, an unit testing tool;
• Test Kitchen, an integration testing tool.

It also includes a host of other Chef tools: Chef Client; Ohai; Knife and Chef Zero. Chef Client is the tool that runs inside a node (i.e., a machine or server) and given a run_list (a set of cookbooks) configures it. Ohai is a tool whose main purpose is to gather the attributes (e.g.: memory, cpu, platform information) of a node and feed them to Chef Client. Knife is a command-line tool that interacts with Chef. Its name should be prefixed with “Swiss Army”… if you’re curious, type knife -h at the terminal. Finally Chef Zero is an in-memory Chef Server, whose main use case is testing.

You might notice that the real Chef Server was not mentioned. Chef Server is a whole other topic and an article in itself, so it won’t be mentioned in the remainder of this article.

We’ll also be using VirtualBox as the virtual machine host environment and Vagrant as its driver. Again, if you want to follow along, go get them now.

With our development environment set up it’s time to create our first cookbook.

Creating our cookbook

Let’s use chef to generate our first cookbook, called my_first_cookbook.

$ chef generate cookbook my_first_cookbook

You’ll notice that chef uses Chef’s recipes to generate your repository skeleton:

Compiling Cookbooks...
Recipe: code_generator::cookbook
  * directory[/Users/joaomiranda/Dev/chef-test/my_first_cookbook] action create
    - create new directory /Users/joaomiranda/Dev/chef-test/my_first_cookbook
  * template[/Users/joaomiranda/Dev/chef-test/my_first_cookbook/metadata.rb] action create_if_missing
    - create new file /Users/joaomiranda/Dev/chef-test/my_first_cookbook/metadata.rb
    - update content in file /Users/joaomiranda/Dev/chef-test/my_first_cookbook/metadata.rb from none to 760bcb
    (diff output suppressed by config)
[...]

Your cookbook will have the following structure:

my_first_cookbook
└── recipes
|   └── default.rb
├── .gitignore
├── .kitchen.yml
├── Berksfile
├── chefignore
├── metadata.rb
└── README.md

Let’s go through each item, one-by-one:

• my_first_cookbook/ - Contains the “my_first_cookbook” cookbook.
• my_first_cookbook/recipes - Contains the cookbook’s recipes.
• my_first_cookbook/recipes/default.rb - The default recipe. It can be seen as the cookbook’s entry point (similarly to main() in Java or C#).
• my_first_cookbook/.gitignore - chef assumes you’ll store your cookbook on Git, so it produces .gitignore to ignore files that shouldn’t be under version control.
• my_first_cookbook/.kitchen.yml - Test Kitchen configuration file.
• my_first_cookbook/Berksfile - Berkshelf’s configuration file. It mainly tells Berkshelf what are the cookbook’s dependencies, which can be specified directly in this file or indirectly through metadata.rb, as we’ll see. It also tells Berkshelf where it should look for those dependencies, usually at Chef’s Supermarket, the cookbook community site.
• my_first_cookbook/chefignore - In the same vein as .gitignore, it tells Chef which files should be ignored when uploading the cookbook to a Chef Server or when sharing them with Chef’s Supermarket.
• my_first_cookbook/metadata.rb - Meta information about you cookbook, such as name, contacts or description. It can also state the cookbook’s dependencies.
• my_first_cookbook/README.me - Documentation entry point for the repo.

That’s a lot of stuff to wrap our heads around! Let’s discuss some of it in more detail, starting with the cookbook. According to Chef’s docs:

A cookbook is the fundamental unit of configuration and policy distribution.

For instance if you need to install nginx on your node, you’ll use a cookbook to do that. There are about 1800 community provided cookbooks at the Supermarket.

A cookbook may contain many different types of artifacts. The most common are the recipes and attributes, which we’ll talk about later. It might also include libraries of custom Ruby code; templates for files to be created/configured on nodes; definitions of reusable resource collections; custom resources and providers or files to be transferred to the nodes under configuration.

Before writing our first recipe, we have to do a very important task: describe our cookbook in metadata.rb. Make sure you set the name of your cookbook and its version. You can add many different pieces of information, but I’d like to highlight that if your cookbook depends on other cookbooks, you are strongly urged to state those dependencies through the use of the depends keyword.

name             'my_first_cookbook'
maintainer       'João Miranda'
maintainer_email 'joao.hugo.miranda@gmail.com'
license          'MIT'
description      'A simple cookbook to illustrate some infrastructure as code concepts'
version          '0.1.0'

depends          'windows', '~> 1.34'

A sample metadata.rb file. Notice how the cookbook depends on the windows cookbook.

Recipes

The next step is to create a recipe. According to Chef’s docs:

A recipe is the most fundamental configuration element within the organization.

Not exactly helpful, right? An example will come to the rescue. For the purpose of this article, we’ll use the hello world of configuration management tools: we’ll install a web server and publish an html page.

If you’re on Red Hat Enterprise Linux (RHEL) or CentOS, place the following inside my_first_cookbook/recipes/default.rb:

package 'httpd'

service 'httpd' do
  action [:enable, :start]
end

file '/var/www/html/index.html' do
  content "<html>
      <body>
        <h1>#{node['index_message']}</h1>
      </body>
    </html>"
end

Hello world with RHEL/CentOS.

Replace 'httpd' with 'apache2' in the previous file if you’re on Ubuntu.

If you’re on Windows, put the following instead:

["IIS-WebServerRole", "IIS-WebServer"].each do |feature|
  windows_feature feature do
    action :install
  end
end

service 'w3svc' do
  action [:start, :enable]
end

file 'c:\inetpub\wwwroot\Default.htm' do
  content "<html>
  <body>
    <h1>#{node['index_message']}</h1>
  </body>
</html>"
end

Hello world with Windows.

Those are very small recipes, but they allow us to touch on several concepts in one swoop. Let’s start with some general considerations before going through the recipes step-by-step.

A crucial property of recipes (and resources) is that they should be idempotent. We should be able to run a recipe any number of times and always get one of two results. Either the node is on its desired state, as specified by the recipes, and it stays that way. Or the node’s state drifted and it is converged to the desired state. Idempotency is a concept that all tools like Chef provide.

You might have noticed that the second and third steps are common both to Linux and Windows, except for the service name and the file paths. Recipes are written in a declarative style and try to abstract away the underlying OS-specific algorithms that converge the node to the desired state. As you’ve seen, there are some differences that have to be accounted for, but it does a good job considering how different operating systems can be.

The recipe’s execution order is determined by reading the recipe top to bottom. Execution order is a contentious theme in the configuration tools community. Some tools, such as Puppet, favour explicit dependencies, where each configuration step declares what other steps need to be executed beforehand. The idea is similar to stating task’s dependencies in build tools such as make or Rake. Others, like Chef or Ansible, favour implicit ordering. This means that in Chef’s case, for instance, order of execution is determined by the order resources are placed in the recipe file.

Resources

So, what are the recipes doing? First of all, they are making sure that the web server is installed:

package 'httpd'

or, on Windows:

["IIS-WebServerRole", "IIS-WebServer"].each do |feature|
  windows_feature feature do
    action :install
  end
end

Both package and windows_feature are resources. A resource describes a desired state you want the node to be in. The important point is that we are describing, or declaring, that desired state, but we are not telling how to get there. package is telling that we want the httpd package installed. windows_feature is telling that we need a Windows Role or Feature installed. Notice how we’re using a standard Ruby array to enumerate the windows_feature resource twice.

The second step is declaring that we need a service (httpd or w3svc) enabled and started. The actions, as specified by action vary from resource to resource.

service 'httpd' do
  action [:start, :enable]
end

The third step is creating a file locally on the node. We are using an attribute, content to specify the file content. Resource can have any number of attributes, which also vary from resource to resource. When the file’s content needs to be dynamically generated, you’re better served with templates.

file '/var/www/html/index.html' do
  content "<html>
    <body>
      <h1>#{node['index_message']}</h1>
    </body>
  </html>"
end

Attributes

This third step also introduces something we haven’t seen before: node['index_message']. What we’re doing here is referencing a node’s attribute. Every node has a set of attributes that describe it. Yes, Chef uses the same word to describe two different concepts: there are resource’s attributes and node’s attributes. They might seem similar at first, both are describing properties of something after all. But node attributes are one of the pillars of a cookbook.

Node attributes are so important that several cookbook patterns rely on them. Node attributes allow for reusable cookbooks, because they make them configurable and flexible. Usually, a cookbook defines default values for the attributes it uses. These default values are placed on Ruby files, inside the cookbook’s attributes folder. This folder is not created upon the cookbook creation, so you have to create it manually. Then you can create a Ruby file, e.g. default.rb, and define attributes like this:

default['index_message'] = 'Hello World!'

Attributes can then be overriden in a number of ways. They can be defined in several places: the nodes themselves; attribute files; recipes; environments and roles. Ohai gathers a host of node attributes automatically: Kernel data; CPU data; platform data; Fully qualified domain names (FQDN); among many other data. Environment (i.e. Dev, QA, Production) attributes are useful to specify data such as connection strings and settings that change from environment to environment. Roles can also have attributes, but even Adam Jacob, Chef’s co-founder, discourages (see the comments) this option.

You can define many types of attributes, in many different places. You can also override attributed values. You have a lot of power in your hands. All this power can make it hard to understand how Chef finds the actual attribute value during a Chef run so make sure you understand the attributes precedence rules.

Providers

Given that resources abstract away the how-to, which piece of Chef’s machinery is responsible for putting a node in its desired state? This is where providers come in. Each resource has one or more providers. A provider knows how to translate the resource definition to executable steps on a specific platform. For instance, the service resource has providers for Debian, Red Hat and Windows, among others. It’s out of the scope of the article to explain how to create you own custom resources and providers, called Lightweight Resource Providers (LWRPs). If you’re interested in learning more, Chef’s docs have an article that shows how simple the process is.

Cookbook Development Process

What we have learned so far enables us to write recipes and thus, configure nodes. We could stop there, but we’re treating “Infrastructure as Code”. We need a development process that allows us to grow while maintaining quality code. Let’s see how we can do that with Chef and its ecosystem.

Among a host of other things, modern development practices include: a build process; linting tools; unit testing; integration testing. We’ll use Rake to define our build process. It’s a simple one, with only four tasks for RuboCop, FoodCritic, ChefSpec and Test Kitchen. The Rakefile, which should be at the cookbook’s root directory (e.g. like metadata.rb), looks like this:

require 'rspec/core/rake_task'
require 'rubocop/rake_task'
require 'foodcritic'
require 'kitchen'

# Style tests. Rubocop and Foodcritic
namespace :style do
  desc 'Run Ruby style checks'
  RuboCop::RakeTask.new(:ruby)

  desc 'Run Chef style checks'
  FoodCritic::Rake::LintTask.new(:chef) do |t|
    t.options = {
      fail_tags: ['any']
    }
  end
end

desc 'Run all style checks'
task style: ['style:ruby', 'style:chef']

desc 'Run ChefSpec examples'
RSpec::Core::RakeTask.new(:unit) do |t|
  t.pattern = './**/unit/**/*_spec.rb'
end

desc 'Run Test Kitchen'
task :integration do
  Kitchen.logger = Kitchen.default_file_logger
  Kitchen::Config.new.instances.each do |instance|
    instance.test(:always)
  end
end

# Default
task default: %w(style unit)

task full: %w(style unit integration)

Let’s review each task, giving a quick tour of each tool and its purpose.

Is our cookbook a good Ruby citizen?

The build process starts by running two static analysis tools: RuboCop and Foodcritic.

RuboCop inspects your Ruby code for compliance with the community Ruby Style Guide. Within Chef’s context, at least recipes, resources, providers, attributes and libraries are Ruby code, so they all should be good Ruby citizens. If you are new to Ruby, RuboCop helps you get up to speed faster, teaching you the way (some) things are done in Ruby.

To see RuboCop in action, let’s assume we are checking our Windows recipe. If we execute chef exec rake at the cookbook’s root directory, RuboCop will break the build and provide this information (you might get additional messages):

Inspecting 1 file
C

Offenses:

s.rb:1:2: C: Prefer single-quoted strings when you don't need string interpolation or special symbols.
["IIS-WebServerRole", "IIS-WebServer"].each do |feature|
 ^^^^^^^^^^^^^^^^^^^
s.rb:1:23: C: Prefer single-quoted strings when you don't need string interpolation or special symbols.
["IIS-WebServerRole", "IIS-WebServer"].each do |feature|
                      ^^^^^^^^^^^^^^^

1 file inspected, 2 offenses detected

Tools like RuboCop can reveal a huge number of violations, especially on code bases that did not use them from the start. RuboCop is configurable: you can switch on or off specific style checks any way you want. You can even tell RuboCop to generate a baseline configuration based on your existing codebase so you do not get overwhelmed with violations.

Your team can also follow some specific guidelines and in that case you can write your own style checks, called custom cops, and plug them into RuboCop.

Are we writing good recipes?

Once you fix all issues found by RuboCop, your recipes will be checked against FoodCritic. FoodCritic has the same kind of role as RuboCop, but while the latter focuses on generic Ruby code issues, the former targets recipe authoring practices.

Let’s temporarily rename metadata.rb to metadata.rb_ and execute chef exec rake again. We should get something like this:

[...]
FC031: Cookbook without metadata file: /Users/joaomiranda/Dev/chef-test/my_first_cookbook/metadata.rb:1
FC045: Consider setting cookbook name in metadata: /Users/joaomiranda/Dev/chef-test/my_first_cookbook/metadata.rb:1

FoodCritic is telling us that we are violating rules FC031 and FC045. “Why does FoodCritic enforce these rules?”, you might ask. Well, one of FoodCritic’s great things is that it gives a clear explanation for each of its rules. For instance, regarding rule FC031, FoodCritic’s docs say the following:

FC031: Cookbook without metadata file
Chef cookbooks normally include a metadata.rb file which can be used to express a wide range of metadata about a cookbook. This warning is shown when a directory appears to contain a cookbook, but does not include the expected metadata.rb file at the top-level.

As with RuboCop, FoodCritic is also configurable. You can turn on or off each rule and you can also create your own rules. Etsy published their own FoodCritic’s rules, for instance.

Static analysis tools are a great addition to your toolbox. They can help you find some errors early and we all know how fast feedback loops are important. These tools also help the newcomer learn about a given language or tool. But I would say that their most important factor is the consistency they promote. As is often said, code is read many more times, by many more people, than it is written. If we promote consistency, the code becomes easier to read as our brain do not has to grapple with the small stuff. It can instead focus on understanding the big picture.

It should be clear that static analysis tools do not have much to say about the design and structure of our code. They may give some hints, but this is the realm of the creative human mind.

Fast feedback with ChefSpec

Static analysis tools, as their name implies, cannot do dynamic analysis. It’s time to turn our attention to unit testing. The Chef’s tool of choice is ChefSpec. ChefSpec is a Chef unit testing framework, built on top of RSpec, meaning it follows the Behaviour-Driven Development school of thought. According to ChefSpec’s excellent docs:

ChefSpec runs your cookbook(s) locally with Chef Solo without actually converging a node. This has two primary benefits:

• It’s really fast!
• Your tests can vary node attributes, operating systems, and search results to assert behavior under varying conditions.

These are very important properties! We can test our code without actually using virtual machines or cloud providers. In order to achieve it, ChefSpec mocks out the providers behind the scenes, so that the configurations are not applied to any possible node.

Let’s write two simple tests to show ChefSpec in action. We’ll start by creating two files that are not strictly needed to run ChefSpec but which helps us to have a fine-tuned environment.

At the root directory of the cookbook, create a file named .rspec with the following content:

--default-path ./test/unit
--color
--format documentation
--require spec_helper

This file sets some options that are passed on to RSpec when it executes. It saves us from having to type them whenever we want to run RSpec. The options we’ve set are:

• Assume the default path to look for examples (tests) is ./test/unit. We’ll understand why in a minute.
• Colorize the output.
• Print RSpec’s execution output in a format that also serves as documentation.
• Automatically require the spec_helper.rb file.

This last item brings us to the second file we must create, spec_helper.rb. Use this file to write code that is needed for all examples (a.k.a tests). Put it inside my_first_cookbook\test\unit, with the following content:

require 'chefspec'
ChefSpec::Coverage.start!
require 'chefspec/berkshelf'

spec_helper.rb is:

• requiring ChefSpec, so that we can use it with RSpec.
• enabling resource coverage, so we can see if we’re touching every resource when the tests are executed.
• telling ChefSpec that we are using Berkshelf so that it can find the cookbook’s dependencies and activate any matchers that it might find.

Finally, let’s create a test for the following resource:

service 'httpd' do
  action [:start, :enable]
end

Create a default_spec.rb file inside my_first_cookbook\test\unit with this content:

describe 'my_first_cookbook::default' do
  let(:chef_run) { ChefSpec::SoloRunner.converge(described_recipe) }

  subject { chef_run }

  it { is_expected.to enable_service('httpd') }
  it { is_expected.to start_service('httpd') }
end

It looks remarkably like English, doesn’t it? We are describing my_first_cookbok´s default recipe. We are simulating a node’s convergence by faking a chef_run, i.e., faking Chef’s execution of a recipe on a node. We are also telling ChefSpec that the subject of our test is the chef_run. We close the description by telling that we are expecting that the chef_run enabled and started the httpd service upon the convergence.

It is important to note that enable_service and start_service are matchers defined by ChefSpec. They are the ones that allow us to assert facts about the recipe’s execution. As always, we can define our own custom matchers, but ChefSpec already includes the most common ones.

If we execute chef exec rake, we’ll get this output:

[...]
my_first_cookbook::default
  should enable service "httpd"
  should start service "httpd"

Finished in 0.0803 seconds (files took 6.42 seconds to load)
2 examples, 0 failures

ChefSpec Coverage report generated...

  Total Resources:   3
  Touched Resources: 1
  Touch Coverage:    33.33%

Untouched Resources:

  package[httpd]                     my_first_cookbook/recipes/default.rb:1
  file[/var/www/html/index.html]     my_first_cookbook/recipes/default.rb:7

You’ll notice that at the start of the output we have English-like sentences. They are directly derived from the tests and can be seen as a specification of what the recipe is supposed to do.

Due to the way Chef works internally, it is not possible to use regular code coverage tools as Seth Vargo, ChefSpec’s author explains. So ChefSpec provides something a bit less exhaustive: resource coverage. We see from the output that the recipe contains three resources, but the tests only touched one. How much coverage is enough? Try to reach at least 80%.

Our cookbook in the real world

Ok, it’s time to exercise our recipe in the real world and do some integration testing. This kind of testing should not find a lot of surprises, if two conditions are met. First, we know how the resources our recipes are using behave in the platform we’re targeting. Second, we’ve written a good set of ChefSpec tests, meaning they cover all the different configuration scenarios our recipes are supposed to handle. With integration testing, running times get an order of magnitude slower, so the more we can do at the unit testing level, the better. But integration testing is where the rubber hits the road. Integration testing allows us to exercise our recipes against real (ok, most likely virtual…) machines.

Integration testing significantly increases the complexity of our infrastructure. Which operating systems do we have to support? Do we support Linux and Windows? Where are our nodes? In the cloud (AWS, Digital Ocean, Azure)? On-premises (e.g.: managed by VMWare’s vSphere)? How many server roles do we have? It quickly gets very complicated.

Fortunately, clever people already grappled with this problem. According to its authors:

Test Kitchen is a test harness tool to execute your configured code on one or more platforms in isolation. A driver plugin architecture is used which lets you run your code on various cloud providers and virtualization technologies such as Amazon EC2, Blue Box, CloudStack, Digital Ocean, Rackspace, OpenStack, Vagrant, Docker, LXC containers, and more. Many testing frameworks are already supported out of the box including Bats, shUnit2, RSpec, Serverspec, with others being created weekly.

So, Test Kitchen is our friend. The idea behind it is very simple. It’s easier to understand it with an example. Our cookbook root directory contains a .kitchen.yml file that looks like this:

---
driver:
  name: vagrant

provisioner:
  name: chef_zero

platforms:
  - name: ubuntu-12.04
  - name: centos-6.4

suites:
  - name: default
    run_list:
      - recipe[my_first_cookbook::default]
    attributes:

This simple file touches on (almost) all the concepts Test Kitchen relies on. It contains a list of platforms, the list of machines where we’ll run our tests. Platforms usually map to a bare bones machine - you’re testing their configuration process, after all - but they can be any kind of machine with any configuration. There is also a list of suites, each specifying a Chef run-list with (optional) attributes definitions. A driver (in our case, Vagrant) manages the platforms. Finally, the provisioner (again, Chef Zero in our case) applies each suite to each platform, unless we explicitly exclude it from the suite. Test Kitchen can thus be seen as an orchestrator. Notice how we haven’t mentioned anything about tests, which might seem a bit weird. We’ll get to that in due time.

Test Kitchen defines a state machine to control its execution. Test Kitchen starts by creating a platform instance by asking the driver to create a virtual machine. It then tells the provisioner to converge the node. After converging the node, Test Kitchen looks for tests and if it finds any, it runs them and puts the instance into the verified state. The cycle closes when Test Kitchen destroys the instance. Given that this cycle can be slow, Test Kitchen helps when things go wrong, by reverting to the last known good state when one of the steps fails. For instance, if a convergence succeeds but a test fails and thus, the verify phase fails, then the instance is kept in the converged state.

So, if we go to the command line and type chef exec rake full, we will eventually run Test Kitchen:

[...]
-----> Cleaning up any prior instances of <default-ubuntu-1204>
-----> Destroying <default-ubuntu-1204>...
       Finished destroying <default-ubuntu-1204> (0m0.00s).
-----> Testing <default-ubuntu-1204>
-----> Creating <default-ubuntu-1204>...
       Bringing machine 'default' up with 'virtualbox' provider...
       ==> default: Box 'opscode-ubuntu-12.04' could not be found. Attempting to find and install...
           default: Box Provider: virtualbox
           default: Box Version: >= 0
       ==> default: Adding box 'opscode-ubuntu-12.04' (v0) for provider: virtualbox
           default: Downloading: https://opscode-vm-bento.s3.amazonaws.com/vagrant/virtualbox/opscode_ubuntu-12.04_chef-provisionerless.box

[...]

  - my_first_cookbook       
Compiling Cookbooks...       
Converging 3 resources       
Recipe: my_first_cookbook::default       
  * package[httpd] action install[2014-11-17T18:50:19+00:00] INFO: Processing package[httpd] action install (my_first_cookbook::default line 1)       


    ================================================================================       
    Error executing action `install` on resource 'package[httpd]'       
    ================================================================================

[...]

A couple of interesting things just happpened. First, Test Kitchen told Vagrant to launch a new machine, defaulting to a box, which you can think as a virtual machine template, provided by Chef: Downloading: https://opscode-vm-bento.s3.amazonaws.com/vagrant/virtualbox/opscode_ubuntu-12.04_chef-provisionerless.box. This box corresponds to the ubuntu-12.04 platform we stated earlier in .kitchen.yml. You can specify your own boxes, of course.

Second, we got an error! Let’s check our Test Kitchen instances and see that default-ubuntu-12.04 is in the Created state, because the convergence step failed.

$ kitchen list
Instance             Driver   Provisioner  Last Action
default-ubuntu-1204  Vagrant  ChefZero     Created
default-centos-64    Vagrant  ChefZero     <Not Created>

We could log in to the instance by doing kitchen login and inspect the machine configuration to find out what went wrong. But the error occurred on the Ubuntu platform and our recipe does not support Ubuntu. We don’t want to support it, so let’s remove the - name: ubuntu-12.04 line from .kitchen.yml.

Let’s move on and execute rake again and this time everything should run smoothly.

[...]
-----> Cleaning up any prior instances of <default-centos-64>
-----> Destroying <default-centos-64>...
       Finished destroying <default-centos-64> (0m0.00s).
-----> Testing <default-centos-64>
-----> Creating <default-centos-64>...
       Bringing machine 'default' up with 'virtualbox' provider...
       ==> default: Box 'opscode-centos-6.4' could not be found. Attempting to find and install...
           default: Box Provider: virtualbox
           default: Box Version: >= 0
       ==> default: Adding box 'opscode-centos-6.4' (v0) for provider: virtualbox
           default: Downloading: https://opscode-vm-bento.s3.amazonaws.com/vagrant/virtualbox/opscode_centos-6.4_chef-provisionerless.box

[...]

     Installing Chef 
       installing with rpm...
       warning: /tmp/install.sh.2579/chef-11.16.4-1.el6.x86_64.rpm: Header V4 DSA/SHA1 Signature, key ID 83ef826a: NOKEY
Preparing...                #####  ########################################### [100%]
   1:chef                          ########################################### [100%]
       Thank you for installing Chef!

[...]

       [2014-11-21T00:47:59+00:00] INFO: Run List is [recipe[my_first_cookbook::default]]
       [2014-11-21T00:47:59+00:00] INFO: Starting Chef Run for default-centos-64
       [2014-11-21T00:47:59+00:00] INFO: Loading cookbooks [my_first_cookbook@0.1.0]
       Synchronizing Cookbooks:
         - my_first_cookbook
       Compiling Cookbooks...
       Converging 3 resources
       Recipe: my_first_cookbook::default
         * package[httpd] action install[2014-11-21T00:47:59+00:00] INFO: Processing package[httpd] action install (my_first_cookbook::default line 1)
       [2014-11-21T00:48:30+00:00] INFO: package[httpd] installing httpd-2.2.15-39.el6.centos from base repository

           - install version 2.2.15-39.el6.centos of package httpd

[...]

[2014-11-21T00:48:50+00:00] INFO: Chef Run complete in 51.251318527 seconds

      Running handlers:
      [2014-11-21T00:48:50+00:00] INFO: Running report handlers
      Running handlers complete
      [2014-11-21T00:48:50+00:00] INFO: Report handlers complete
      Chef Client finished, 4/4 resources updated in 58.012060901 seconds
      Finished converging <default-centos-64> (2m4.75s).

Altough a successful Chef run tells us a lot, especially when we have the ChefSpec safety net, we can add an additional layer of testing. Test Kitchen does not provide a testing framework, so it is unable to execute automated tests by itself. Test Kitchen relies on already existing test frameworks, such as Bats, shUnit2, RSpec or Serverspec. We’ll use ServerSpec to write a simple test.

ServerSpec is built on top of RSpec, just like ChefSpec, but their mechanics are completely different. While ChefSpec has an intimate knowledge of Chef and its inner workings, ServerSpec has no idea that Chef even exists. ServerSpec just makes assertions about a machine’s state: Is this package installed? Is that service enabled? ServerSpec has no idea how that package was installed or the service enabled. For all it cares, those operations could have been performed manually!

Let’s create a very simple test. Create a file named default_spec.rb inside my_first_cookbook\test\integration\default\serverspec, with the following contents:

describe package('httpd')do
  it { should be_installed }
end

The directory structure follows some specific conventions:

• test\integration - Test Kitchen looks for tests here.
• default - This is the exact same name of the suite we want to test
• serverspec - This tells Test Kitchen to use ServerSpec as its test framework.

If we execute chef exec rake full again, Test Kitchen will find our test and execute it:

[...]
-----> Verifying <default-centos-64>...
       Removing /tmp/busser/suites/serverspec
       Uploading /tmp/busser/suites/serverspec/default_spec.rb (mode=0644)
-----> Running serverspec test suite
[...]       
       Package "httpd"
         should be installed

       Finished in 0.0623 seconds (files took 0.3629 seconds to load)
       1 example, 0 failures
       Finished verifying <default-centos-64> (0m1.54s).
-----> Kitchen is finished. (0m3.95s)

The test succeeds because ServerSpec asserted that the package is indeed installed on that instance. The ChefSpec equivalent would only assert that the package Chef resource had been touched.

When shall we write ServerSpec tests? When shall we write ChefSpec tests? That’s material for a whole new article. I’d suggest that the Test Pyramid can be applied to infrastructure testing as well, so you should have a larger number of ChefSpec tests. Actually, before writing integration tests with ServerSpec, or a similar framework, ask yourself if your ChefSpec tests and successful Chef runs already cover your validation needs.

We’ve seen Test Kitchen’s work on Linux. What about Windows? Unfortunately, at the moment Test Kitchen does not support Windows. But there is hope! Salim Afiune is working on bringing that support and Matt Wrock wrote an article where it shows how you can use Test Kitchen with Windows today. There are some additional rough edges, when you have medium to large tests, but they can be overcome.

Wrapping it up

We know the basic concepts of Chef. We know how to harness tools to help our Chef development process. What’s next? Actually, quite a lot. Chef is a (very) large product. I hope I gave you a head start. Start small. Make sure you fully understand Chef’s concepts. Create some cookbooks. Reuse some cookbooks. You’ll be a Master Chef in no time.

About the Author

João Miranda (@jhosm) started his career in 2000, at the height of the dot-com bubble. That enlightening experience led him to the conclusion that agile practices are the best way to respond to the business needs of almost all organizations. He currently is a principal software engineer at OutSystems, a PaaS provider, where he helps to remove all friction that may hinder the development teams’ fast pace.