Living In A Microservice World: Why Kubernetes Security Matters

By Flux7 Labs
March 22, 2019

Kubernetes Security Best Practices Microservices

This article originally appeared on Forbes 

It’s frequently said that Kubernetes is one of the fastest-growing projects in open-source history (paywall). Along with containers and microservices, Kubernetes is gaining traction within more and more enterprises as it helps expedite time to market, more quickly meeting evolving customer demands while providing greater return on investment with less total cost of ownership. Yet, amid all the enthusiasm for increased productivity, it’s important not to forget about security controls in the process.

Just as DevOps has helped build momentum for microservice technologies, DevSecOps can help proactively address gaps in security before they occur. By working hand-in-hand across Development, Operations and Security, an enterprise can proactively establish security with agility, which I define as a combination of guardrails, automation and best practices.

As use of Kubernetes grows so will bad actors looking to take advantage of organizations still getting their arms around cloud-based Kubernetes clusters. The best approach is to take a holistic security view, ensuring that Kubernetes-based microservices receive the same scrutiny from your security team, but likely with even stricter controls and rules than the rest of your environment given the multiple levels of security within Kubernetes that need to be addressed.

Your security team should understand the unique, multidimensional security issues surrounding Kubernetes. Security will need to establish policies and controls that protect host machines, containers and the control plane — not just from outside attacks, but also from each other. As teams often jump into working with Kubernetes without first embracing security best practices or fully understanding its complexities, it’s vital that your security team understands that a compromise of the control plane will result in all the organization’s containers becoming vulnerable. Best practices will help ensure, for example, that a compromised application won’t threaten other containers.

With a hat tip to Levi Blaney, DevOps Engineer at Flux7, here are six key areas to start a best practices conversation with your development and security teams to ensure Kubernetes security controls are embedded in your DevSecOps strategy:

1. Role-Based Access Controls (RBAC)

RBAC is a common security approach across most enterprises, assigning access to resources based on a person’s role within the organization. Many companies pair RBAC with the Principle of Least Privilege (PoLP) in which individuals are only able to access the resources necessary to conduct their assigned duties.

In Kubernetes, application programming interfaces (APIs) are the central interface for admins, users, applications and service accounts to initiate operations. As a result, applying RBAC and PoLP to control API access is imperative to keeping systems safe and to ensuring audit requirements are met. In addition in Kubernetes, roles are not assigned by default to service accounts. Yet, service accounts are necessary to manage Kubernetes clusters. As a result, it’s not uncommon to find organizations granting cluster-wide access to all resources as a shortcut to cluster management. Using Kubernetes’ authentication, authorization and admission controls in conjunction with Security policies will help DevSecOps avoid shortcuts and achieve meaningful access controls.

2. Pod Security Context

A Kubernetes pod is a group of containers deployed together on the same server. For this group, a security context will define privilege and access controls, providing the necessary framework to ensure that the pod — and containers within it — have the privileges to only access the resources needed and no more. Moreover, security context allows admins to control who can create resources by limiting capabilities to a given role or group. Pod security should be set to meet your organization’s specific security policies.

3. Resource Isolation

A pod is just one example of how an organization can isolate resources; clusters, namespaces and nodes can also be used, limiting sharing of resources to decrease risk. For example, you might want to avoid co-locating payment card industry (PCI) and non-PCI workloads for regulatory compliance reasons.

Kubernetes offers features like namespaces that separate tenants and their Kubernetes resources into their own namespace. From here, Security can help develop policies to be applied across isolation units, effectively restricting access, resource usage and more, helping prevent denial-of-service attacks while providing data protection.

4. Network Security

Further to the point about resource isolation is network security. Enterprises should have established network controls for Kubernetes namespaces, pods and other resources such that cross-talk is restricted to meet security’s network policy. Additionally, quota and limit ranges can be used to control access to ports and load balancers, which will affect their visibility outside a cluster. Network security can secure API access, helping keep threats from compromised containers, misuse and misconfigurations at bay.

5. Secret Management

Like other application platforms, Kubernetes works with secrets — those things that should remain tightly controlled, like API keys and passwords. Secrets should never be hardcoded; use automation to encrypt and keep secrets safe, injecting them into DevOps pipelines when needed. Whether you use the Kubernetes mechanism for secrets or another solution, it is imperative that it be configured correctly as secrets in the wrong hands could wreak serious havoc.

6. Verify With Logging And Auditing

“Trust but verify” is an old adage in security circles that helps assure system security by verifying that preventative, detective and corrective controls operate as expected. At my firm, we refer to these tools as inspectors. Inspectors are automated tools that monitor, log and introspect services. While Kubernetes offers Audit Logging for this purpose, recording actions taken by the API for auditing, other inspectors include solutions like Splunk and CIS Benchmarks.

While Kubernetes is rapidly evolving, it’s important to stay on top of security best practices as they emerge. For example, giving secure access to a service in one container without giving it to all containers is still an emerging challenge that hasn’t been solved across platforms. With an active community, solutions to challenges like these are rapidly emerging. With Development, Operations and Security working in tight coordination to develop policies and controls, the power of Kubernetes can be reaped in an environment that is both agile and secure.