Docker Fans Rejoice! CoreOS Added to DigitalOcean

Public cloud provider DigitalOcean recently announced support for CoreOS, the new Docker-centric Linux distribution. CoreOS is growing in popularity as container-based development goes mainstream, and DigitalOcean is part of a growing pool of providers that support this unique operating system.

In simultaneous blog posts, both DigitalOcean and CoreOS touted their new relationship. The CoreOS blog pointed out that over 2,000 developers had asked for CoreOS on Digital Ocean, and each hosting provider has access to alpha, beta, and release versions.  DigitalOcean announced that the alpha channel of CoreOS could be deployed to one of their virtual containers called “droplets.” In addition to CoreOS, Digital Ocean supports other Linux distributions including CentOS, Debian, Fedora, and Ubuntu.

CoreOS is a lightweight Linux distribution – based on Google’s Chrome OS – that is optimized for clusters, not individual servers. It runs only a handful of base services, doesn’t have a package manager, and expects nearly everything to run within Docker containers. Cluster management is done using a pair of core technologies that DigitalOcean described in a new tutorial. Service discovery across the cluster is enabled through a service called etcd.

The etcd service is a highly available key-value store that can be used by each node to get configuration data, query information about running services, and publish information that should be known to other members. Each node runs its own etcd client. These are configured to communicate with the other clients in the cluster to share and distribute information.

Applications wishing to retrieve information from the store simply need to connect to the etcd interface on their local machine. All etcd data will be available on each node, regardless of where it is actually stored and each stored value will be distributed and replicated automatically throughout the cluster. Leader elections are also handled automatically, making management of the key-store fairly trivial.

Orchestration of a CoreOS cluster and Docker containers is done by a service called fleet.

A rather simple concept, fleet acts as a cluster-wide init system.

Each individual node within a clustered environment operates its own conventional systemd init system. This is used to start and manage services on the local machine. In a simplified sense, what fleet does is provide an interface for controlling each of the cluster members' systemd systems.

You can start or stop services or get state information about running processes across your entire cluster. However, fleet does a few important things to make this more usable. It handles the process distribution mechanism, so it can start services on less busy hosts.

You can also specify placement conditions for the services you are running. You can insist that a service must or must not run on certain hosts depending on where they are located, what they are already running, and more. Because fleet leverages systemd for starting the local processes, each of the files which define services are systemd unit files (with a few custom options). You can pass these configuration files into fleet once and manage them for the entire cluster.

CoreOS has a unique auto-update strategy that applies any available OS patches to a dual root partition.

Initially, your system is booted into the root A partition and CoreOS begins talking to the update service to find out about new updates. If there is an update available it is downloaded and installed to root B. To ensure we don’t disrupt your application, we rate limit the disk and network I/O this process is allowed to use with Linux cgroups.

Using this dual-root scheme is an improvement on the existing workflow of yum or apt-get. Using these tools during upgrading has been known to cause the package manager to force daemons to use new libraries or move configuration files around. With CoreOS, a system update is an atomic operation that can be rolled back.

On CoreOS, the root partition you are running against isn’t modified and your server is never in an unstable or partially upgraded state. To finish off the upgrade, reboot the machine and in a few seconds you will start running on root B with a freshly updated system.

Cloud providers have been lining up to support CoreOS. In addition to DigitalOcean, CoreOS is available on AWS, Google Compute Engine, CenturyLink Cloud, Rackspace, OpenStack,, VMware, and more. The open source Panamax project from CenturyLink Labs (see previous InfoQ coverage here) bakes CoreOS into its Docker management platform and uses fleet for orchestration.

Pricing for DigitalOcean droplets starts at $5 per month for a single CPU, 512 MB of memory, and a 20 GB SSD drive. DigitalOcean has a global presence with data centers in San Francisco, New York, London, Amsterdam, and Singapore. Their launch of CoreOS support was accompanied by a set of tutorials for creating CoreOS clusters and working with Docker containers. It sounds like more is on the way from the Digital Ocean team, according to TechCrunch.

As the company’s co-founder Ben Uretsky has often told me, the company was so busy just scaling up to meet demand over the last few years that basic product development like adding more distributions often fell by the wayside. Now that DigitalOcean has plenty of funding and a larger staff, it’s able to focus more on adding new features like today’s CoreOS launch.