Docker Images and you
- An Image is a "base", read only virtual machine template and a "Container" is a writable instance of that machine;
- Images can be chained in a hierarchy of inheritance where a Base image is modified generating a child image and so on. This is "the way" to build images, even though you can always build your own.
- If a base image has a security bug, it is at least possible (if not likely) that all the children will inherit the same bug;
- The logic with which developers most likely approach this model is "I won't have to worry about the base image". This has been somehow hinted at and while some experienced developers will take the need for updates into consideration, not everyone will;
- People will download and build upon a set of images from Docker's repository. No, we will not hack that, stop being evil!
Yeah, OK. So, Shellshock and Docker?
Things I have learnt scanning the 100 most popular Docker images
- 30% of the top 100 images were still vulnerable to one of the shellshock bugs;
- 4 of the top 30 were vulnerable, 1 in the top 10 - so around 10% of the really popular images;
- None of the vulnerable images were "official, Docker maintained images", but some were based on them: those images were still vulnerable because they were not rebuilt after the patch had been applied on the base images. That is, using a base image that gets regularly updated is not enough;
- Some of the vulnerable images have a consistent user base, or at least downloads. asher/remote_syslog has got almost 900.000 downloads;
- Docker security team is really nice. I gave them an heads up (nothing for them to do here really, in terms of incident response, but a lot of long term work) and they were very direct about the issues and shared some nice insights. Thumbs up.
|A synthetic summary of the shellshock related bugs I've found scanning on October 9th 2014|
Things you should worry about
However I think there are some subtle but important differences here. As an admin or security person, you can't just SSH in the machine and "apt-get upgrade" it, then save a new snapshot. There is a whole chain of images that might get forked in various points, where some of the nodes might even be escaping your control. Updating images is a very real, known problem: the Docker security team told me they are looking into it so hopefully things are going to get better in the future, but for now you really want to have a story for managing updates. Possibly before the next Shellshock.
My humble view on things that could be improved
- Reporting bugs on Docker images is hard! Some of these images have tens of thousands of users but no bug tracker or no clear way to report security bugs. In some cases I've opened an issue in github and hoped for the best. Providing some kind of built-in bug reporting feature would be a nice to have in the registry, or maybe this could be brewed in Dockerfiles?
- Old images are bad! When you look at an image in Docker's repository you have a clear indication of when it was built (or at least committed). Check out the Properties of itzg/minecraft-server: it has been built before the Shellshock bug was even discovered and it's based on an official base image. Now, given that we know what base images are vulnerable to bugs and when, it should be possible to simply assume that all the images that have been built before that as potentially vulnerable as well;
- Custom images are a lot of work to maintain. On one of my bug reports the maintainer of the image just said "sure, I'll rebuild". Since he was using an official Debian build as a base image, it's not a lot of work on his side. Had he used a completely custom OS, he'd have to do a standard upgrade, which takes more and more time and effort as the image ages.
EDIT: I have been pointed this blog post which does a detailed analysis of some of the official Base Images - I have only pulled the Latest tag for each Image, so they got more coverage there. From a quick skim, none of the images I've found to be vulnerable were based on the images they flag in the article.