Demystifying Docker and Kubernetes Security
What is Docker
Docker is a tool designed to make it easier to create, deploy, and run applications by using containers. Containers allow a developer to package up an application with all of its necessary parts such as libraries and other dependencies and ship it all out as one package that can be run on any operating system platform.
While this sounds just like a virtual machine running in Parallels, VMWare, Hyper-V, etc, it is quite different in the way it interacts with the host operating system it's running on. In a virtual machine, each guest host runs its own operating system, using its own kernel, contrary to containers, which uses the kernel running in the host OS. This requires the developer to only package the container with whatever the host computer is not already running giving it a significant performance boost over virtual machines. It also eliminates the virtual device drivers created by hypervisors for the different devices running in the guest OS.
Here is a diagram I've created illustrating those differences between VMs and containers.
What is Kubernetes
Kubernetes manages containers, facilitating both configuration and automation. While containers are a good way to bundle and run applications, as containerized workloads and services grow beyond just a few to dozens if not hundreds, you need a management system that can handle administration and scaling with resiliency. Kubernetes addresses downtime concerns, fail-over, deployment patterns, service discovery and load balancing, storage orchestration, automated roll-outs and rollbacks, automatic bin packing, self-healing, secret and configuration management, and more.
It's important to remember that Kubernetes is not monolithic, meaning, it operates at the container level rather than at the hardware level, so it isn't a traditional platform-as-a-service (PaaS).
Adversarial Goals Targeting Containers
When an adversary is targeting a container, her ultimate objective is to break out of the container environment and execute code on the host OS with superuser privileges (root).
Additionally, adversaries will want to pivot to other container hosts in the network once they've escaped the container to the host OS. The common attack vectors that affect containers include:
Kernel exploits
Container breakouts
Denial of Service (DoS) attacks
Poisoned Images
Because of the way containers are designed, there have been previous examples where it was possible to leverage host-level vulnerabilities in things like RunC which allowed researchers to break out of the container.
This specific RunC vulnerability allows a malicious container (with minimal user interaction) to overwrite the host RunC binary and thus gain root-level code execution on the host. The level of user interaction is being able to run any command (attacker controlled or not) as root within a container in either of these contexts:
Creating a new container using an attacker-controlled image; and
Attaching (docker exec) into an existing container, which the attacker had previous write access to.
This vulnerability is not blocked by the default AppArmor policy, nor by the default SELinux policy on Fedora (because container processes appear to be running as container_runtime_t). However, it is blocked through correct use of user namespaces (where the host root is not mapped into the container's user namespace). The exploit for this particular vulnerability is available here.
Hardening Docker Containers
A series of hardening steps can be taken to make Docker containers more resilient against attack, which should also be combined with any number of different security solutions available from vendors like Aqua Security, SysDig, Snyk, and Neuvector. However, below is a general list of hardening steps that can be taken:
Ensure that you have a vulnerability and patch management program in place to address vulnerabilities in your container environment;
Perform system and network monitoring and audit logs along with other container activity;
Enable Docker content trust: Docker Content Trust is a new feature incorporated into Docker 1.8. It is disabled by default, but once enabled, it allows you to verify the integrity, authenticity, and publication date of all Docker images from the Docker Hub Registry;
Run the Docker Bench: Once the script is run, you will notice a lot of information regarding configuration best practices for deploying Docker containers that can be used to further secure your Docker server and containers;
Secure Secrets: Sensitive information such as secrets (e.g., SSH keys, passwords, tokens, TLS certificates) need to be encrypted and stored in a Secrets Manager (e.g., Docker Swarm, HashiCorp Vault) and not at the host level;
Access Control: Having a well-established access management solution for Docker is a must so that containers can operate with minimal privileges and access, which helps reduce risk. Large organizations can incorporate role-based access control (RBAC) and use directory solutions such as Active Directory to manage permissions for all personnel of the organization;
Docker Image Authentication: always use base images that are reviewed and scanned by Docker’s Security Scanning Services, or use a base image that is digitally signed by Docker Content Trust;
To reduce performance impacts and denial-of-service attacks, it is a good practice to implement limits on the system resources that the containers can consume;
Least privileged user: Create a dedicated user and group on the image, with minimal permissions to run the application and use the same user to run this process;
Sign and verify images to mitigate MITM attacks: We put a lot of trust into docker images. It is critical to make sure the image we’re pulling is the one pushed by the publisher, and that no one has tampered with it;
Address open source library vulnerabilities: Scan your docker images for known vulnerabilities and integrate it as part of your continuous integration. Snyk is an open source tool that scans for security vulnerabilities in open source application libraries and docker images;
Use copy instead of add: Arbitrary URLs specified for ADD could result in MITM attacks, or sources of malicious data. In addition, ADD implicitly unpacks local archives which may not be expected and result in path traversal and Zip Slip vulnerabilities;
Use a linter: Enforce Dockerfile best practices automatically by using a static code analysis tool such as hadolint linter, that will detect and alert for issues found in a Dockerfile;
Use Docker containers with SELinux enabled (--selinux-enabled). This prevents processes inside the container from overwriting the host docker-runc binary;
Use read-only file system on the host, at least for storing the docker-runc binary; and
Use a low privileged user inside the container or a new user namespace with uid 0 mapped to that user (then that user should not have write access to runc binary on the host).
Analyst Insight
In Q3 I will be publishing research reports on API security and container security solution vendors, performing product reviews, writing case studies, and market reports. I've already covered a few API security vendors and published reports on them located at https://aitegroup.com/users/alissa-knight
Comment
So what's your opinion? Do you follow any other Kubernetes or Docker hardening steps or use some tools or products you can talk about? Leave your comments in the section below!
Like & Share
As usual, if you liked this article, please support me by clicking LIKE and share it with your own feed! This is the best possible way that you can support me and my continued research. If anyone has anything to add or comment on in this article, please feel free to share it with everyone below in the comments section! Learn more about me at my homepage at www.alissaknight.com, LinkedIn, watch my VLOGs on my YouTube channel, listen to my weekly podcast episodes, or follow me on Twitter @alissaknight.