Concourse, an open source CI pipeline tool that uses yaml files for configuring pipelines and configuration-free setup, has recently bumped its major release and is currently available in version 1.1.0. According to the team sponsored by Pivotal, the major benefits of Concourse are explicit and first-class support of pipelines, running isolated builds in containers, avoidance of snowflake build servers and easy access to build logs.
Conceptual simplicity is an additional advantage claimed by Concourse:
Concourse's end goal is to provide an expressive system with as few distinct moving parts as possible. Concourse limits itself to three core concepts: tasks, resources, and the jobs that compose them.
Tasks are basic units of execution, i.e. scripts that run inside of a freshly started container. The container is prepared so that it contains an input and an output directory for the task script to run on.
Resources are abstract locations of versioned artifacts such as repositories and are used to model external dependencies that enter or exit a pipeline and more abstract concepts such as time triggers. Changes can be detected (check), fetched (get) and published (put). The different resource types (such as git, AWS S3 or triggers) encapsulate boiler plate code away from pipelines and provide a pluggable interface for extending Concourse.
A job composes resources and tasks with the help of a build plan. Jobs are triggered by changes in resources or manually for example for human approvals. The snippet below illustrates a simple job:
jobs:
- name: hello-world
plan:
- task: say-hello
config:
platform: linux
image_resource:
type: docker-image
source: {repository: ubuntu}
run:
path: echo
args: ["Hello, world!"]
An instance of execution of a job's plan is called a build. Builds in Concourse are reproducible since their tasks run afresh in new containers. As a result, build workers are not polluted with changes from previous runs. Moreover, if a build fails, it can be rerun locally within a container with Concourse CLI called Fly, enabling faster development cycles. In addition,
The containers running in a build can be accessed while they're running (and also shortly after they finish) via fly intercept, which can greatly help in debugging.
Pipelines are visualisations of the resulting flow of resources through jobs. There can be multiple pipelines for a deployment that are isolated from each other. They support GoCD like fan-in and fan-out and are fully configured in a yaml file:
As the Concourse team report, the motivation for building yet another CI tool resulted from the dissatisfaction with existing tooling. For example, Jenkins depends on plugins and click-based configuration, making instances difficult to recreate and with GoCD “finding how to configure something is very difficult and the execution hierarchy is deep and complex.”
While Concourse can be used for CI and CD, currently available resources lack some more deployment-focused implementations, leaving continuous delivery and deployment task implementation to users. In addition, the current dashboard doesn't provide a lot of operational information. However, since metrics are essential in understanding how systems are behaving and performing, “Concourse can emit metrics about both the system health itself and about the builds that it is running. Operators can tap into these metrics in order to observe the health of the system.” Concourse’s own pipeline and metrics are exposed to the public.
Concourse utilises some Pivotal / Cloud Foundry lingo and tools such as the Garden API for abstracting running containers or BOSH for managing releases. So being involved in that ecosystem does help in getting started with Concourse. To bridge that greater initial skill gap, online documentation and some tutorials are available.