Chef Cookbook Management

How to manage community and personal cookbooks in a git version controlled chef-repo.

Give me a path, Vasily. One path only, please...

It used to be recommended that we customize cookbooks' templates, files, recipes, etc by expecting knife cookbook upload to merge multiple cookbook paths set in knife.rb cookbook_path (e.g. cookbooks, site-cookbooks) but that is being deprecated. You can see this by installing a cookbook in both paths and then see the warning given when trying to upload the cookbook to a Chef server.

All of this leads me to use a single path chef-repo/cookbooks for cookbook_path.
This makes it the authoritative cookbook location whether running Chef-Solo in a Vagrant VM or using knife to upload cookbooks to the Chef server.

Librarian

Librarian is by far the best way I've found to bundle unmodified community, modified community, personal and dependency cookbooks.
It provides me all the flexibility I need where other methods failed.
I've described the other methods below to help explain what the other methods lacked.

Librarian especially makes contribution of fixes and new cookbooks to the community much easier.

I chose to install Librarian a little different from the recommendations in Librarian's documentation.

• I don’t integrate Librarian with knife.rb because it breaks any knife command unless run from within the chef-repo directory so it sees the Cheffile.
• The Librarian author seems to expect the use of site-cookbooks for customizing cookbooks but as I describe above that is deprecated.

My Chef-Repo is an example of how I use Librarian.

Manage Unmodified Community, Personal and Dependency Cookbooks

Install Librarian in the chef-repo/ directory.

Do NOT use the chef-repo/cookbooks directory to track your cookbooks because it is now managed by Librarian.

Source any unmodified cookbooks in Librarian's Cheffile.
You don't have to source a cookbook's dependencies because Librarian will automatically find them for you.

Manage Modified Community and Personal Git Hosted Cookbooks

Create a cookbooks directory just outside of chef-repo.
Clone forked community cookbooks and git hosted original cookbooks that you want to modify and track.
This makes it easy to contribute fixed cookbooks and new cookbooks to the community.

Use :path to source these cookbooks in the Librarian Cheffile during testing.
Use :git to source them after committing and pushing the changes.

Manage Personal Cookbooks Not Already Version Controlled

Create a chef-repo/site-cookbooks directory for tracking original cookbooks within the chef-repo.
When running the knife cookbook create [new_cookbook] command be sure to add -o ./site-cookbooks so the cookbook doesn't get created in the default chef-repo/cookbooks directory.

Do NOT add the chef-repo/site-cookbooks directory to your knife.rb cookbook_path.

Use :path to source the cookbook in Librarian's Cheffile.

Prepare Cookbooks for Vagrant or Knife Use

cd chef-repo
librarian-chef clean
librarian-chef install

Alternatives for Managing Customized Community and Personal Cookbooks

• knife cookbook github install
  knife cookbook github install is in my opinion a better option than the traditional knife cookbook site install for installing/updating community cookbooks that I want to customize. But after using this method I decided to stop using it in favor of the approach I mention above.

  • Primary point of failure is that it tracks cookbook modifications in your chef-repo even though these cookbooks have their own git repos.
    • You can not easily contribute changes back to the community.
    • You don't have full access to the cookbook's git repo.
  • This isn’t capable of auto fetching a cookbook's dependencies but that's a good thing if you want Librarian to manage dependencies.
  • This can fetch from arbitrary github users/repos/branches which is much more flexible.
  • This can show diff between local cookbook and remote cookbook using github compare feature
  • This merges the most recent commit from github instead of the most recent version as specified by the maintainer meaning you can be more up to date.

• knife cookbook site install
  Opscode is recommending the use of knife cookbook site install for streamlining install, tracking, customizing of community cookbooks.

  • Primary point of failure is that it tracks cookbook modifications in your chef-repo even though most of these cookbooks have their own git repos.
    • You can not easily contribute changes back to the community.
    • You don't have any access to the cookbook's git repo.
  • You only have access through http://community.opscode.com/cookbooks API which is pretty limiting.
  • Sometimes the http://community.opscode.com/cookbooks versions are not as up to date as the github repos which means you have to wait for someone to make a new release to get a fix or feature that is already in github.
  • If you use this method along with Librarian (described above) you should ALWAYS use ‘knife cookbook site install -D’ to make sure you do NOT install cookbook dependencies; instead manage dependencies using Librarian Cheffile