It was exactly 04:12 AM. I woke up to my phone vibrating with a 'Critical' alert. In one of our testCompany production clusters, I saw a simple, internet-facing 'customer-feedback' pod doing something it should never do: it was trying to query the entire list of 'secrets' via the kube-apiserver. While rubbing my eyes, my first thought was clear: someone is inside, and they are currently trying to move laterally through the cluster.
Even though I look at things from a Red Teamer's perspective, figuring out what was happening inside that pod that night turned into a total detective story. Most folks assume Kubernetes (K8s) is secure right out of the box, but in reality, we have a massive orchestra here—and if the conductor is drunk, the concert becomes a disaster.
The First Hole: Default ServiceAccounts and 'Over-Generous' Roles
The attacker's first stop that night was the ServiceAccount token embedded within the pod. In Kubernetes, every pod is born with a service account unless specified otherwise. If you don't properly restrict the permissions of this pod, the moment an attacker breaches it, they grab that token and start asking the cluster, 'Who am I and what can I do?'
The first command the attacker used probably looked something like this:
# Checking permissions with the token inside the pod
KUBE_TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)
curl -k -H "Authorization: Bearer $KUBE_TOKEN" https://127.0.0.1:6443/api/v1/namespaces/default/secrets
Under normal conditions, this request should return a '403 Forbidden'. But in our scenario, the dev team—probably to 'keep things moving fast'—had defined a role similar to cluster-admin for this service account. And that's where the nightmare begins.
RBAC: Don't Be Afraid to Be Stingy with Permissions
Role-Based Access Control (RBAC) is where the most mistakes happen in Kubernetes. Usually, when creating a ClusterRoleBinding, the logic follows 'Let’s give it all permissions for now, we'll fix it once it works' instead of 'What specific permissions does it actually need?'.
Here is an example of a malicious ClusterRoleBinding (Defanged):
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: feedback-pod-admin-binding
subjects:
- kind: ServiceAccount
name: default
namespace: production
roleRef:
kind: ClusterRole
name: cluster-admin # THIS IS WHERE WE HIT THE WALL
apiGroup: rbac.authorization.k8s.io
The cluster-admin role gives the attacker 'God Mode' over the cluster. If a pod only needs to read configmaps in its own namespace, but you give it permission to see secrets across the entire cluster, you're essentially leaving the keys in the door.
The Defense Recipe:
- Always use the
automountServiceAccountToken: falsesetting. If your pod doesn't need to talk to the API, there is no reason to mount that token inside. - Apply the 'Least Privilege' principle. Define only the necessary verbs (get, list) and the necessary resources (pods, deployments).
Escaping the Container: hostPath and Privileged Pods
In that night's crisis, the attacker didn't stop at the API. They took it a step further to try and escape to the node itself...