Getting to know Kubernetes vulnerability management

When it comes to open-source software (OSS) like Kubernetes, some people get nervous not knowing the people who have worked on the code in the project. Contrary to popular belief, many OSS projects have robust security teams and rigorous vulnerability management processes, just like you’d expect to find in proprietary software. 

For Kubernetes, a dedicated Product Security Committee oversees the security response process. The Product Security Committee is a group of core maintainers, many with security-specific roles, nominated by other core maintainers and technical advisors within the community. The Committee’s role is to respond to any and all emails about a potential vulnerability, according to a documented response process. Here’s an overview.

1. Triage the disclosure 

When the team receives a disclosure, it begins investigating whether the submission is a real issue or just a bug without security implications. If the committee confirms that it’s an issue, it then leads the development and release of a patch and notifies the community. 

2. Assess the impact of the vulnerability

One key initial step is to determine a vulnerability’s potential impact. This is usually represented as a Common Vulnerability Scoring System (CVSS) score and the documented severity criteria for Kubernetes vulnerabilities. This score looks at criteria like how easily the issue can be exploited, the consequences if it is exploited, and the privileges required to exploit it. A score under 4.0 is considered low; between 4.0 and 7.0 is medium; between 7.0 and 9.0 is high and above 9.0 is critical. This CVSS score acts as a rough barometer for how issues should be prioritised and with what urgency, with the caveat that the vulnerability’s severity can vary depending on the specific Kubernetes deployment configuration and environment.

3. Generate a fix

Next, the Product Security Committee works with the relevant developers to generate a fix. If the vulnerability involves components from other open-source projects, the team works with security groups within those projects. 

There are times when a security fix can happen in the open, as part of a normal patch release. But if the vulnerability is severe enough, the patch will be developed and tested in private. At this point, only those who need to know should be aware of the vulnerability; you wouldn’t want someone with malicious intent to have early knowledge of an unpatched issue.

For fixes that follow the private release process, the release of a security patch will include an explanatory announcement to the kubernetes-security-announce community. When necessary, a retrospective or postmortem is also released to the Kubernetes community, spelling out the steps taken, response timeline and any other relevant details. 

4. Roll out the fix

The next step is to roll out notifications to certified Kubernetes distributors. Because distributors are responsible for what end-users receive, it’s important that they have the opportunity to patch early and prepare their own notifications. Like many open-source projects, Kubernetes has a private list of distributors who get embargoed security notifications; anyone who meets the criteria can sign up for these messages. 

So what does all this mean for you as a Kubernetes user

First of all, make sure your team knows about the Product Security Committee’s process. It should help anyone who’s unsure about open source security feel more confident in Kubernetes. The Product Security Committee is a robust and responsive group of Kubernetes and security experts working on your behalf, complete with an on-call rotation during working hours. 

Second, if you’re deploying a container on Kubernetes and you notice something that makes you suspect a security issue, you should notify the Product Security Committee right away, safe in the knowledge that it’ll follow a prescribed process to get to the bottom of things. For more information about when and how to report a vulnerability, check out Kubernetes Security and Disclosure Information. 

It’s also important to know your provider’s communication policies with regard to vulnerabilities. To that end, be sure you know the answer to the following questions:

• If there’s a problem, when will my provider contact me?
• Where are security bulletins posted for the services I’m consuming?
• How can I find out when patches will be available for those services?
• Where can I find information about vulnerabilities for what I run in my environment?

When appropriate, Google Cloud publishes security bulletins for Google Kubernetes Engine (GKE) and sends email notifications to affected users. GKE users should defer to the GKE security bulletins as their source of truth; it’s updated whenever a new issue is severe enough to warrant attention.

In short, as a Kubernetes end-user you should:

• Share the Kubernetes response and reporting processes with your team
• Report any future suspected issues through the Kubernetes disclosure process
• Learn about your provider’s disclosure and patching policies

By answering these questions and familiarising yourself with the Kubernetes security response process, you can be an informed and confident user of open source software and ensure that your organisation is taking an active role in your container security.