Kubernetes Penetration Test Report: Insights and Twistlock Response

This post is also available in: 日本語 (Japanese)

The Cloud Native Computing Foundation (CNCF) late last year commissioned a penetration test to identify unknown security vulnerabilities and design weaknesses in Kubernetes. The final report is posted in the working group’s repository.

When done well, penetration tests provide methods for improving software security quality. The Kubernetes test was thorough and well designed. It resulted in dozens of findings, including identification of many new vulnerabilities and recommendations that new security feature enhancements be implemented. The Kubernetes project just opened #81146 as a single tracker to follow progress of the 37 issues that were identified. 

We’ve received many questions about this report from our customers. Answers to the most common questions are posted below. We will update this post as more information becomes available.

1. Why was the report released prior to all issues being addressed?

Disclosure of the report findings wasn’t announced or broadly coordinated prior to its release. While the community was aware of the penetration test, full disclosure of results, including unfixed vulnerabilities, was a surprise. In the spirit of improving everyone’s security, Ariel Zelivansky, who leads our research team, opened #3982 in the community repo to discuss developing more coordinated processes for disclosure and proactive remediation going forward. Future vulnerability findings will ideally be coordinated among maintainers and vendors so there are already fixes available for vulnerabilities at the time of disclosure.

2. Does Twistlock detect these new vulnerabilities yet?

Our Intelligence Stream consumes vulnerability data from dozens of vendors and upstream providers to build the data set used to identify vulnerabilities within each customer environment. Since this report was unexpectedly released, many of these vulnerabilities do not yet have CVEs assigned and thus many vendors have not yet had an opportunity to assess whether their distributions are vulnerable. As vendors conduct these assessments and publish their results, the Intelligence Stream will automatically pick up these vulnerabilities and enable Twistlock to detect them. Vendor analysis of newly disclosed vulnerabilities typically happens quickly, often within hours of disclosure. In this case, though, it may take a little longer for some of the lower severity findings to be evaluated and for CVE data to be published about them due to the sheer volume of the findings.  

No customer action is needed. As soon as vendors publish vulnerability data, it will immediately be picked up by the Intelligence Stream and used to detect vulnerabilities in your environment.

3. How does Twistlock protect me from these vulnerabilities?

Twistlock includes a variety of different controls that mitigate findings in the report. Of the 37 findings, only five are rated as high impact. Of these five, one is a suggested enhancement, rather than a vulnerability. We examine those five findings below.

4. How does Twistlock protect against these issues?

hostPath PersistentVolumes enable PodSecurityPolicy bypass

Twistlock provides two mitigations: First, our Kubernetes audit monitoring alerts on pods created with additional privileges (accessing host mounts). Second, we have a compliance rule that alerts / blocks in cases where pods are created with host mounts (this is compliance check #55 within Twistlock).

Kubernetes does not facilitate certificate revocation

This is the suggested security enhancement. Re-keying certificate chains is overly burdensome. The finding recommends simplifying it. While we agree that this would be a useful security advancement, it’s not a vulnerability and doesn’t create any direct risk to users today. As this is a suggested platform improvement, it’s outside the scope of our focus as a security product.

HTTPS connections are not authenticated

The core risk here is the eventual access to etcd. Pods don’t normally access etcd directly and as our Cloud Native Network Firewall automatically learns normal traffic patterns, we’d see and block this anomalous connection. Further, the attack requires creating a malicious kubelet, which we help customers mitigate with our Trusted Images feature that only allows software to run from approved registries and repositories.

TOCTOU when moving PID to manager’s cgroup via kubelet

In this vulnerability, the process inside a container eventually writes to devices on the host. Our runtime defense feature already automatically detects and prevents these types of attacks. Twistlock will automatically learn that this is not normal file system access behavior and automatically prevent it.

Improperly patched directory traversal in kubectl cp

This is CVE-2019-1002101, which Ariel found earlier this year. You can read about it in our original blog post. The CVE involves a malicious image (which can be prevented with Trusted Images) and can be mitigated by requiring a read only rootfs, for which we provide a compliance check.