blog.6admin.io
guest@6admin.io:~$
hardening, kubernetes, security

Hardening Kubernetes

Kubernetes multiplies the single-container problem by a control plane, a flat network and an authorization system and each of them ships a default that is convenient rather than safe. The keys to the whole cluster are three: a pod that can reach the host, an RBAC grant that is too broad and a network that lets every pod talk to every other. Harden those three and the exotic attacks lose their first step. Leave one open and the rest of the hardening is decoration, because a privileged pod, a cluster-admin binding or a reachable etcd are already game over.

Pods: stopping one from becoming the node

A pod that runs privileged, mounts a host path, or shares the host PID or network namespace is not isolated from the node and the node runs the kubelet credentials that reach the cluster. Pod-level controls are therefore where a container compromise is kept from becoming a node compromise:

securityContext:
  runAsNonRoot: true
  allowPrivilegeEscalation: false
  readOnlyRootFilesystem: true
  capabilities:
    drop: ["ALL"]
  seccompProfile:
    type: RuntimeDefault

The dangerous specs are the mirror of that: privileged: true, hostPID/hostNetwork: true and above all a hostPath volume mounting /, /var/lib/kubelet or /etc/kubernetes, each of which hands the pod the node’s filesystem and with it the cluster certificates. Enforce the right shape rather than hoping for it, with the built-in Pod Security Admission set to restricted on every namespace that has no documented reason to be looser:

# namespace label - admission rejects non-compliant pods
pod-security.kubernetes.io/enforce: restricted

The Pod Security Standards replaced the old PodSecurityPolicy and restricted encodes most of the securityContext above as an admission rule, so a developer cannot ship a privileged pod into that namespace by forgetting the context. For what the built-in levels cannot express, a policy engine (Kyverno, OPA Gatekeeper) is the next layer, but start with the built-in one, which is free.

RBAC: the grant is the blast radius

RBAC is where clusters quietly go wrong, because an over-broad grant is invisible until someone uses it. The rules are blunt: nobody and nothing gets cluster-admin without a proven need, no Role uses a wildcard verb or resource and a handful of verbs are escalation primitives that deserve their own review. create pods in a namespace with a privileged service account is node access; get secrets is every credential in the namespace; and impersonate, escalate and bind let a principal grant itself more than it has, turning a modest foothold into any role it can name.

A service account gets a Role scoped to the exact resources and verbs its workload calls, bound in its own namespace and nothing cluster-wide unless it truly operates cluster-wide:

kind: Role
rules:
- apiGroups: [""]
  resources: ["configmaps"]
  verbs: ["get", "list"]      # not "*", not "secrets"

Default service-account tokens are the other quiet exposure: a pod that does not call the API should not carry a token it can leak, so set automountServiceAccountToken: false by default and opt in where the API is actually used.

Network: the default is allow-all, so default-deny

The flat pod network is the default nobody remembers: with no NetworkPolicy in place, every pod can reach every other, across namespaces, including the services near the control plane and the databases. A single compromised front-end can then scan ports and reach the whole inside of the cluster. The fix is a default-deny policy per namespace, then explicit allowances for the flows that must exist:

kind: NetworkPolicy
metadata: { name: default-deny, namespace: app }
spec:
  podSelector: {}
  policyTypes: ["Ingress", "Egress"]   # deny both until a policy permits

This only works if the CNI enforces NetworkPolicy (Calico, Cilium do; some do not), something to verify rather than assume. Once default-deny is in place, a compromised pod only reaches what you explicitly allowed it, which is the segmentation the flat network otherwise refuses.

Secrets and etcd: base64 is not encryption

A Kubernetes Secret is base64-encoded, not encrypted and by default it sits in etcd in that form, which means access to etcd, or a backup of it, is every secret in cleartext. Two things follow. Turn on encryption at rest so the API server encrypts secrets before they touch etcd, ideally with a KMS provider rather than a local key:

# EncryptionConfiguration referenced by the API server
resources:
- resources: ["secrets"]
  providers:
  - kms: { name: cluster-kms, endpoint: ... }
  - identity: {}

And keep secrets out of the places that leak them: not in environment variables (readable by the API and any in-pod RCE), not in ConfigMaps (not secret at all) and increasingly not in etcd at all but pulled at runtime from an external store through the Secrets Store CSI driver. etcd itself is the jewel, so it gets peer and client TLS, its own firewalled network and encrypted backups, because a readable etcd backup is a cluster compromise that outlives the cluster.

The control-plane edges

Two points deserve an explicit check because their unsafe settings have been real incidents. The API server must not allow anonymous authentication to anything meaningful and the kubelet is a second API easy to leave under-authenticated: --anonymous-auth=false and --authorization-mode=Webhook on the kubelet close the path where a pod talks to its node’s kubelet and gets command execution on every pod there. Audit logging on the API server is the record of who did what and it is off or minimal by default.

Images close the loop as for plain Docker: scan in the pipeline, sign with cosign and verify at admission so an unsigned or vulnerable image cannot be scheduled, rather than discovering it already running.

Where to land

Enforce Pod Security restricted on every namespace so no pod reaches the node through privilege or a hostPath. Scope every RBAC grant to the exact verbs and resources, keep cluster-admin nearly empty and watch secrets, impersonate, escalate and bind as the escalation verbs they are. Lay down a default-deny NetworkPolicy in every namespace on a CNI that enforces it, because the flat network is allow-all until you say otherwise. Encrypt secrets at rest or pull them from an external store and protect etcd as the cleartext copy of everything it is. Then ordinary container and image hardening on top. The pattern in all of this is the same: Kubernetes ships an open default and every control here is you closing a precise door it left ajar.