Opscode released Chef 11 early this month with enhancements to its scalability to meet the demands of hyperscale web operations. Opscode rewrote the entire server core API in Erlang and at the same time kept it backward compatible. Opscode renamed the core server API "Erchef" to complement the rewrite in Erlang. They have also separated the server source code from the client and have decomposed the server into multiple component projects, each written in Erlang.
Seth Falcon wrote a follow-up blog post where he provided additional background on Chef 11 including:
The features of Erlang which make it a good fit for high volume web services are its share-nothing memory and process model, multi-core scalability, and the soft-realtime performance resulting from Erlang’s per-process garbage collection model.
Additionally Timothy Prickett Morgan wrote about how the following improvements are based on the rewrite in Erlang:
Chef 11 has an order of magnitude reduction in memory footprint compared to the Ruby version in Chef 10.
Chef 11 can manage up to 10,000 nodes from a single server, which is a factor of four more than the Chef 10 server could handle.
The web user interface remains in Ruby for Chef 11, however it now runs on the Rails 3.2 framework instead of Merb and interacts with "Erchef" through its API. Omnibus packaging now makes it possible to distribute the core server API and GUI as single package in the native package management systems for multiple platforms including: Enterprise Linux [x86_64] (v. 5,6) and Ubuntu [x86_64] (v. 10.04, 10.10, 11.04, 11.10, 12.04, 12.10). Opscode will also be releasing migration tools for existing Chef 10 open source server users.
Opscode has added many improvements to client tools in Chef 11 including modified functionality in the following areas: attributes, security, remote file mirrors, partial templates, lock file, and LWRPs. However, with the improvements and new features there are some breaking changes. Engineers can deploy the server component of Chef 11 and run the Chef 10 clients until the cookbooks are upgraded to meet the requirements for the client component of Chef 11.
Additional improvements to the Chef 11 client tool suite include:
- Knife command line tool now includes knife-essentials.
- Knife ssh returns an exit code based on remote command execution.
- Shef has been renamed "chef-shell".
- A new tool "chef-apply" is included for running single cookbooks.
Chef 11 continues to be available as an open source product, however now it also has support available from Opscode.