BanyanOps have published a report stating that ‘Over 30% of Official Images in Docker Hub Contain High Priority Security Vulnerabilities’, which include some of the sensational 2014 issues such as ShellShock and Heartbleed. The analysis also looks at user generated ‘general’ repositories and finds an even greater level of vulnerability. Their conclusion is that images should be actively screened for security issues and patched accordingly.
Banyan looked at every image (around 960) from all of the official repos (around 75) looking for vulnerable contents against the Mitre database of Common Vulnerabilities and Exposures (CVE). The discovered vulnerabilities were then graded against distribution specific and US National Vulnerability Database (NVD) assessments to determine if their impact was high, medium or low.
In a post on his personal blog ‘Someone said that 30% of the images on the Docker Registry contain vulnerabilities’ Docker’s Jérôme Petazzoni has provided further analysis and insight on the topic. Petazzoni points out that the source for official images is readily available, meaning that it can be easily assessed and improved upon. Both Banyan and Petazzoni make some effort to explain that some images (e.g. CentOS 5.11) are deliberately held back for the sake of compatibility, and come with stern health warnings stating that security updates should be applied. This means that whilst a large number of the images present might be vulnerable, the weighting on images being used will be quite different.
One issue that isn’t directly addressed is how vulnerabilities can cascade from one image to another due to layering (and the use of FROM within Dockerfiles). This means that a vulnerability in say debian:jessie will also be duplicated in nginx:latest. As many user generated images are also based on official images any underlying vulnerability can then spread further through the dependency graph. CenturyLink recently launched their ImageLayers tool to allow analysis of the construction of images, but it will take some adaptation to be useful for vulnerability analysis since it tends to show the base layer as an ‘ADD file’ and doesn’t include a time dimension.
In a recent interview Ericsson’s Head of Cloud Jason Hoffman stated “Docker’s taking off because it’s the new package management”. That provides a good explanation of Docker’s rapid adoption, but also hides the fact that Docker images are generally dependent on the package manager provided by an underlying Linux distribution. This creates a ‘manifest problem’ in that images become a point in time snapshot of what package managers put into them, and to the extent that a Dockerfile provides a manifest of contents it isn’t detailed enough to be used for vulnerability assessment. Petazzoni provides some direction on how purposeful manifests might be created post hoc, whilst Banyan advocate image scanning (perhaps self servingly if they’re building a platform to do just that).
Another question for Docker users is whether a vulnerability actually affects a running container spawned from a given image? The common practice of shipping around entire Linux distributions in order to support a single running process means that much of an image won’t actually ever get loaded into memory. Heartbleed only matters if OpenSSL is being used, and ShellShock can only be a problem if bash is invoked, though the doctrine of better safe than sorry argues that it’s unsafe to have vulnerabilities lying around waiting to be exploited. It can be difficult to know what’s used at runtime, though ‘How I shrunk a Docker image by 98.8% – featuring fanotify’ shows how extraneous files might be stripped away. The ‘FROM scratch’ pattern of deploying statically linked Golang binaries via containers with no base import also deals with container bloat and extraneous potentially vulnerable files (at the cost of losing memory efficiency that might be gained by dynamic library loading and increasing dependency opacity).
The issue of images and security patches was already addressed in the recently published Docker Security Benchmark. Docker have now also released a Docker Bench for Security script to automate testing of hosts against the benchmark. Further tools to help users assess the security of their images seem likely to follow. Given the nature of Docker Hub as a microcosm of software delivery in general it seems likely that a large number of images there will remain vulnerable; the key action for Docker users will be doing their best to ensure that they aren’t continuing to use images that are vulnerable, and that patches are being applied in a timely manner.