6. Containers, Kubernetes and Serverless Security

Introduction and Context

Cloud-native computing has changed the security problem from protecting a small number of long-lived servers to protecting large populations of short-lived, automated workloads. Containers package applications and their dependencies into portable units. Kubernetes schedules and manages those units across clusters. Serverless platforms execute functions in response to events without requiring the customer to manage the underlying operating system. These technologies improve deployment speed, scalability, and operational efficiency, but they also create new security dependencies.

A container image may contain vulnerable libraries, embedded credentials, excessive operating-system privileges, or malicious code. A Kubernetes cluster may be weakened by an exposed control plane, permissive role-based access control, unsafe admission policies, or unrestricted network communication. A serverless function may have a small code footprint but still possess excessive permissions, process untrusted input, or leak sensitive data through logs. In each case, the apparent ephemerality of the workload can mislead organisations into assuming that it is difficult to attack or unimportant to protect.

Security must therefore be applied throughout the workload lifecycle: source code, dependency management, image construction, registry storage, deployment, runtime operation, monitoring, and retirement. The central principle is that orchestration and automation do not replace security governance; they make security policy more important because a single configuration can affect hundreds or thousands of workloads.

Learning Objectives

After studying this chapter, students should be able to:

1. Explain the security architecture and threat surfaces of containers, Kubernetes, and serverless computing.

2. Distinguish image, registry, runtime, orchestration, and function-level security controls.

3. Apply namespace isolation, admission control, network policy, workload identity, and runtime detection.

4. Use Kubernetes security benchmarks and risk-based assessment methods.

5. Evaluate ephemeral-workload risks, including supply-chain compromise and excessive privilege.

6. Design practical security approaches for cloud-native applications and assess their organisational implications.

The Cloud-Native Workload as a Security Object

A traditional virtual machine generally has a persistent operating-system identity, storage volume, and network location. A container is more transient. It shares the host kernel while isolating processes, filesystems, and resources through operating-system mechanisms. This sharing makes containers lighter than virtual machines, but it also means that a kernel vulnerability or container escape can affect the host and neighbouring workloads.

An image is a read-only template consisting of application code, runtime components, libraries, and metadata. At execution time, a container runtime creates a writable layer above the image. Images are commonly built in layers, which improves reuse but can preserve sensitive material in earlier layers even when it has been deleted from the final filesystem. A secure image is therefore not merely one that produces a successful build; it must be minimal, traceable, reproducible, scanned, signed, and continuously reassessed.

A registry stores and distributes images. Registries are security-critical repositories rather than simple file stores. An attacker who modifies a trusted image can compromise every environment that pulls it. Registry controls consequently include strong authentication, access separation, encryption, immutable tags or digests, vulnerability scanning, malware analysis where appropriate, retention policies, and audit logging.

The container runtime translates orchestration instructions into operating-system processes. Runtime risks include privileged containers, host filesystem mounts, access to the container engine socket, excessive Linux capabilities, unsafe system calls, resource exhaustion, and exploitation of runtime or kernel vulnerabilities. Security profiles, seccomp restrictions, Linux security modules, read-only filesystems, dropped capabilities, and resource limits reduce these risks.

Kubernetes Security Architecture

Kubernetes consists of a control plane and worker nodes. The API server is the principal entry point for cluster administration and workload management. The scheduler selects nodes for pending workloads, while controllers maintain the desired state. The data store, commonly etcd, contains cluster configuration and often secrets. Worker nodes run the kubelet, container runtime, and network components.

This architecture creates several trust boundaries. The API server must authenticate and authorise users, service accounts, and automation systems. Communication between control-plane components and nodes must be authenticated and protected. The kubelet must not be unnecessarily exposed. Etcd requires encryption and strict access control because compromise can provide broad control over the cluster. Network plugins must enforce intended communication rather than merely provide connectivity.

Kubernetes security is frequently described through four interacting planes:

• Supply-chain plane: source code, dependencies, build systems, images, and registries.
• Control plane: API server, authentication, authorisation, admission, audit, and cluster configuration.
• Workload plane: pods, containers, service accounts, secrets, storage, and application behaviour.
• Node and network plane: hosts, kernels, runtime engines, ingress, service discovery, and east–west traffic.

A weakness in any plane can undermine controls in the others. For example, excellent network policies cannot compensate for a compromised image that already contains an application backdoor.

Namespaces and Isolation

Kubernetes namespaces provide logical separation for resources such as pods, services, roles, and configuration objects. They are useful for separating teams, environments, or applications, but they are not equivalent to a complete security boundary. Cluster-scoped resources, node-level privileges, shared infrastructure, and misconfigured role bindings may cross namespace boundaries.

Namespaces should be combined with quotas, default-deny network policies, resource limits, separate service accounts, controlled admission rules, and carefully designed role bindings. Production and development workloads should not share sensitive infrastructure merely because they have different namespace names. Where strong isolation is required, organisations may use dedicated node pools, virtual clusters, hardened runtimes, or separate clusters and accounts.

Authentication, Authorisation, and RBAC

Authentication establishes identity; authorisation determines permitted actions. Kubernetes role-based access control (RBAC) grants permissions to subjects through Roles or ClusterRoles and their bindings. Least privilege requires avoiding broad wildcard permissions, limiting access to specific resources and verbs, and separating human administration from workload identities.

Service accounts should be created for specific applications rather than using a highly privileged default account. Tokens should be short-lived where possible, projected only into workloads that need them, and prevented from being used as general-purpose credentials. A permission to create pods can be highly consequential because it may allow an attacker to mount secrets, impersonate service accounts, or access sensitive nodes. RBAC analysis must therefore consider combinations of permissions rather than isolated verbs.

Admission Control and Policy as Code

Admission occurs after an API request has been authenticated and authorised but before the object is persisted. Admission controllers can reject, modify, or validate requests. They are a powerful preventive layer because they stop unsafe workloads before scheduling.

Useful policies may require signed images, approved registries, non-root execution, dropped capabilities, read-only root filesystems, resource limits, trusted labels, and restricted host access. Policy engines can express these requirements as code and apply them consistently across clusters. However, policies must be tested carefully. Overly permissive rules create gaps; overly strict rules encourage workarounds or disrupt legitimate deployments. A mature approach uses version-controlled policies, staged enforcement, exception workflows, ownership, and audit evidence.

Network Policies and Service Exposure

Kubernetes networking often permits broad pod-to-pod communication unless network policies restrict it. A compromised workload can therefore scan or attack other services even when ingress controls are strong. Network policies should express both ingress and egress intent. A practical design begins with default denial, then permits only required application paths, DNS resolution, observability endpoints, and approved external services.

Network policies are not a substitute for application authentication or encryption. They reduce reachability, while identity and cryptographic controls verify who is communicating. Ingress controllers, load balancers, service meshes, and cloud security groups must also be assessed because traffic may bypass an expected policy path.

Secrets and Workload Identity

Secrets should not be embedded in images, source repositories, command lines, or static configuration files. Kubernetes Secret objects provide organisation and access control, but base64 encoding is not encryption. Encryption at rest, strict RBAC, external secret managers, rotation, and audit logging are necessary for sensitive values.

Workload identity allows a workload to obtain cloud permissions through a bound identity rather than a long-lived access key. The identity should be associated with a specific service account and limited to the actions required by the application. Identity federation reduces credential distribution, but it does not eliminate risk: a compromised workload may still abuse every permission assigned to its identity. Identity design must therefore follow least privilege, audience restriction, short token lifetime, and separation of duties.

Securing the Container Supply Chain

Container security begins before an image exists. Development teams should use trusted base images, pin dependencies, maintain software bills of materials (SBOMs), and scan source code and third-party packages. Builds should occur in controlled environments with minimal credentials and reproducible inputs. Build systems themselves are high-value targets because compromise can produce apparently legitimate malicious images.

Image scanning identifies known vulnerabilities, malware indicators, secrets, and configuration weaknesses. Scanning is valuable but imperfect: databases may be incomplete, vulnerabilities may be unreachable in the deployed application, and a clean scan does not prove that the software is trustworthy. Risk decisions should consider exploitability, exposure, runtime privileges, compensating controls, and the availability of fixes.

Image signing and verification establish provenance and integrity. A digest identifies a precise image, whereas a mutable tag such as “latest” may point to different content over time. Deployment pipelines should verify signatures, provenance attestations, approved builders, and vulnerability thresholds. Registries should prevent unauthorised overwriting, record pull activity, and remove obsolete or unapproved images.

Runtime Detection and Response

Preventive controls cannot eliminate all attacks. Runtime security observes process execution, file access, network connections, privilege changes, container creation, and interactions with the host. Detection rules might identify a shell launched in a normally web-only container, unexpected access to sensitive paths, cryptocurrency-mining behaviour, reverse shells, or attempts to contact the container runtime socket.

Runtime telemetry should be correlated with Kubernetes audit logs, cloud identity records, image metadata, and network flow data. Response actions may include isolating a workload, revoking its identity, stopping or replacing the pod, quarantining the image, preserving forensic evidence, and investigating the build pipeline. Because containers are short-lived, evidence can disappear quickly. Centralised logging, immutable audit storage, node-level telemetry, and rapid alert processing are essential.

Replacement is often safer than attempting to repair a compromised container in place, but replacement alone is insufficient. If the image, deployment manifest, secret, or identity remains compromised, the new workload will reproduce the incident.

Kubernetes Security Benchmarks and Assessment

Kubernetes security benchmarks provide structured checks for control-plane configuration, node hardening, authentication, logging, network settings, and workload practices. They support repeatable assessment and comparison, but compliance with a benchmark is not identical to security. A benchmark may not understand the organisation’s business processes, application architecture, threat model, or compensating controls.

An effective assessment combines benchmark checks with threat modelling and exposure analysis. It should identify internet-facing services, privileged workloads, sensitive data paths, identity relationships, administrative access, and likely attacker objectives. Findings should be prioritised according to exploitability and consequence. For example, an exposed API server with anonymous access is generally more urgent than a low-severity package vulnerability in an isolated, non-sensitive workload.

Assessment should be continuous. Cluster configuration changes rapidly through infrastructure-as-code and deployment automation; periodic manual audits alone cannot provide reliable assurance. Drift detection, policy reports, penetration testing, red-team exercises, and incident simulations complement automated checks.

Serverless Function Security

Serverless functions are event-driven units that execute under a provider-managed runtime. Their reduced infrastructure responsibility does not remove security obligations. Developers remain responsible for code, dependencies, input validation, permissions, data handling, and configuration.

The event source is a major trust boundary. Functions may process HTTP requests, message queues, object-storage events, schedules, or database changes. Events must be authenticated where appropriate, validated against a schema, and protected against replay, injection, oversized payloads, and malicious object names. A function triggered by a storage upload should not automatically trust the uploaded content.

Function identities should be distinct and narrowly scoped. A function that reads from one storage location should not have unrestricted access to an entire account. Environment variables and configuration stores must be protected, and secrets should be retrieved through an approved secret-management mechanism. Dependencies should be pinned and scanned because a small function can still incorporate a large and vulnerable dependency tree.

Cold starts, concurrency, retries, and asynchronous execution create distinctive operational risks. A failed function may be retried and produce duplicate transactions unless the operation is idempotent. Excessive concurrency can exhaust downstream systems. Logs may contain personal data or tokens, while distributed execution can make investigation difficult. Timeouts, concurrency limits, dead-letter queues, structured logging, tracing, and data-minimisation practices are therefore security as well as reliability controls.

Ephemeral-Workload Risk and Governance

Ephemerality changes evidence, identity, and accountability. A pod may exist for seconds, an image may be replaced repeatedly, and a function may execute only during a brief event. Traditional asset inventories based on fixed hostnames are inadequate. Organisations need workload metadata, immutable identifiers, deployment history, image digests, ownership information, and centralised telemetry.

Automation also creates blast-radius concerns. A flawed deployment template can introduce an insecure service account or privileged setting across an entire cluster. Separation of duties, peer review, protected branches, deployment approvals for high-risk changes, and policy-as-code reduce this risk. Emergency access should be time-limited and audited.

Governance must address data residency, privacy, retention, third-party services, software licensing, breach notification, and regulatory requirements. Serverless and managed Kubernetes services create shared-responsibility boundaries that must be documented. The provider secures the underlying service, while the customer remains responsible for configuration, identities, data, application logic, and often workload security.

Illustrative Case Study: Securing a Retail Kubernetes Platform

Background

A fictional online retailer operated a Kubernetes platform for its web application, payment integration, inventory service, and analytics workloads. Development teams could deploy independently, and the cluster used a shared container registry.

Situation and Problem

A security review found that containers ran as root, several teams used the default service account, network traffic was unrestricted between namespaces, and image tags were mutable. The cluster also lacked centralised audit retention. Although no compromise had been confirmed, the platform had a large potential blast radius.

Analysis

Threat modelling identified three high-priority paths. First, a vulnerable public-facing application could be used to obtain a pod shell. Second, the compromised pod could access the Kubernetes API using its default service account. Third, broad network reachability could permit movement toward inventory and payment services. Mutable image tags also meant that a compromised build could be deployed without a clear change record.

Approach and Solution

The organisation introduced separate service accounts and removed unnecessary token mounting. RBAC permissions were rewritten around application functions rather than team convenience. Admission policies required approved registries, signed images, non-root execution, dropped capabilities, resource limits, and read-only filesystems except where explicitly justified.

Namespaces received default-deny ingress and egress policies, followed by narrowly defined application communications. Images were referenced by digest, vulnerability thresholds were enforced in the pipeline, and registry write access was separated from deployment access. Etcd and audit logs were protected, while runtime monitoring alerted on unexpected shells, privilege changes, and host access. A benchmark-based assessment was repeated after remediation, and exceptions received named owners and expiry dates.

Outcome

The measures reduced unauthorised reachability and made deployment provenance easier to establish. Some legacy applications required redesign because they assumed root access or unrestricted networking. Rather than weakening the controls, the organisation used a staged migration and documented temporary exceptions.

Learning Points

The case demonstrates that Kubernetes security is a system of mutually reinforcing controls. Namespaces alone would not have prevented compromise; neither would image scanning alone. Identity, admission, network segmentation, runtime visibility, and governance had to operate together. It also shows that security controls may reveal architectural debt that must be addressed through application engineering rather than configuration changes.

Common Challenges and Mistakes

Frequent mistakes include treating containers as miniature virtual machines, trusting image scans as proof of safety, using the latest tag, granting cluster-admin privileges for convenience, exposing the Docker or container runtime socket, and assuming that a namespace is a hard security boundary. Other errors include creating network policies without testing egress, storing credentials in images, failing to rotate tokens, ignoring control-plane logs, and deleting compromised workloads before collecting evidence.

Serverless programmes commonly overtrust event sources, use one broad execution role for many functions, log sensitive payloads, ignore dependency vulnerabilities, and neglect retry or idempotency behaviour. In both Kubernetes and serverless environments, teams may optimise for deployment speed while leaving ownership unclear. Security responsibility must be embedded in platform engineering, application development, operations, and governance rather than assigned only to a central security team.

Emerging Developments

Important developments include confidential containers, stronger workload attestation, software-provenance frameworks, automated SBOM analysis, eBPF-based runtime monitoring, policy engines integrated with continuous delivery, and finer-grained workload identity. Service meshes may provide mutual TLS and traffic policy, although they add operational complexity. AI-assisted code and configuration generation may increase productivity but also introduce insecure manifests, undocumented dependencies, and unclear provenance; generated artefacts require the same review and testing as human-authored artefacts.

The strategic direction is toward continuously verified workloads: identity is short-lived, software provenance is attestable, policies are evaluated automatically, and runtime behaviour is compared with an expected profile. This approach does not eliminate judgement. It requires risk-based exception handling, clear ownership, and skilled interpretation of automated findings.

Best Practices for Practitioners

A defensible cloud-native security programme should:

• Maintain an inventory of images, functions, clusters, identities, owners, and data flows.
• Build minimal images from trusted and maintained bases; pin dependencies and generate SBOMs.
• Sign images, deploy by digest, protect registries, and verify provenance at admission.
• Enforce non-root execution, dropped capabilities, read-only filesystems, resource limits, and restricted host access.
• Use least-privilege RBAC and workload identity with short-lived credentials.
• Apply default-deny network policies and validate both ingress and egress.
• Centralise audit, runtime, identity, network, and deployment telemetry.
• Test policies and incident procedures continuously, including evidence preservation for ephemeral workloads.
• Treat benchmark compliance as a baseline rather than a complete risk assessment.
• Review serverless event trust, dependency security, retries, concurrency, logging, and data minimisation.

Chapter Summary

Containers, Kubernetes, and serverless functions provide powerful mechanisms for packaging, orchestrating, and executing cloud applications, but their automation and ephemerality create distinctive security challenges. Effective protection spans the supply chain, registries, runtimes, control plane, workloads, identities, networks, and monitoring systems. Kubernetes security depends on carefully designed RBAC, namespace use, admission policies, network controls, secrets management, workload identity, and hardened nodes. Container images require provenance, vulnerability management, signing, and controlled deployment. Serverless functions require secure event handling, least-privilege execution roles, protected secrets, dependency management, and controls for retries and concurrency. Benchmarks support consistent assessment, while threat modelling and continuous monitoring provide context. The most mature approach combines preventive policy, rapid detection, strong governance, and risk-aware operational decision-making.

Key Terms

Container: An isolated process environment that shares the host kernel while packaging application code and dependencies.

Container image: A layered, usually immutable template used to create containers.

Registry: A repository that stores and distributes container images.

Container runtime: Software that creates and manages running containers.

Kubernetes: An orchestration platform that automates deployment, scaling, and management of containerised workloads.

Namespace: A Kubernetes logical partition for organising and controlling resources.

RBAC: Role-based access control that assigns permitted actions to identified subjects.

Admission control: Validation or mutation of Kubernetes API requests before persistence.

Network policy: A rule governing permitted network connections to or from workloads.

Workload identity: A machine identity assigned to a workload for controlled access to services.

Service account: A Kubernetes identity used by applications or automation.

Image digest: A content-derived identifier for an exact image version.

SBOM: A software bill of materials listing components and dependencies in software.

Image signing: A cryptographic method for establishing image integrity and provenance.

Runtime detection: Monitoring of workload behaviour during execution.

Serverless function: An event-driven program executed on provider-managed infrastructure.

Ephemeral workload: A short-lived workload whose identity and evidence may change rapidly.

Admission policy: A codified rule that permits, rejects, or modifies deployment requests.

Security benchmark: A structured set of recommended configuration and control checks.

Review Questions

Short Answer Questions

1. Why is a container image digest preferable to a mutable image tag for deployment?

2. What security purposes do Kubernetes namespaces serve, and what can they not guarantee?

3. Distinguish authentication from authorisation in Kubernetes.

4. Why is base64 encoding insufficient protection for Kubernetes secrets?

5. Identify two security risks specific to serverless function retries or concurrency.

Long Answer Questions

1. Explain the major trust boundaries in a Kubernetes cluster and the controls appropriate to each.

2. Describe a secure container-image lifecycle from source code through registry storage and deployment.

3. Discuss how RBAC, workload identity, admission control, and network policies work together to reduce Kubernetes risk.

4. Explain why ephemeral workloads create challenges for asset inventory, incident response, and accountability.

5. Compare the security responsibilities of a customer using Kubernetes with those of a customer using serverless functions.

Analytical / Application Questions

1. A pod running a public web service can access every service in the cluster and uses a service account with broad cloud permissions. Analyse the likely attack paths and propose a prioritised remediation plan.

2. A serverless image-processing function receives files from an object-storage bucket and occasionally processes the same file twice. Design security and reliability controls for input validation, identity, retries, logging, and downstream access.

3. A development team argues that vulnerability scanning makes image signing and admission control unnecessary. Evaluate this argument and recommend a supply-chain security model.

22. Multiple Choice Questions with Answers

Question 1

Which control most directly prevents an unsigned or unapproved container image from being deployed to a Kubernetes cluster?

A. Network address translation

B. Admission control policy

C. Increased pod memory

D. A mutable image tag

Correct Answer: B. Admission control policy

Question 2

Why is a Kubernetes namespace insufficient as a complete security boundary?

A. Namespaces cannot contain services

B. Namespaces eliminate RBAC

C. Cluster-scoped and node-level privileges may cross namespace boundaries

D. Namespaces prevent all network communication

Correct Answer: C. Cluster-scoped and node-level privileges may cross namespace boundaries

Question 3

Which practice provides the strongest assurance that a deployment refers to one precise image version?

A. Using the latest tag

B. Using an image digest

C. Renaming the image

D. Storing the image on a public registry

Correct Answer: B. Using an image digest

Question 4

A compromised pod attempts to access the container runtime socket mounted from the host. What is the principal concern?

A. Improved DNS performance

B. Possible control of the host or other workloads

C. Automatic image signing

D. Reduced application latency

Correct Answer: B. Possible control of the host or other workloads

Question 5

What is the primary purpose of a default-deny Kubernetes network policy?

A. To permit all traffic unless encrypted

B. To block all workloads permanently

C. To require explicit approval for necessary network paths

D. To replace application authentication

Correct Answer: C. To require explicit approval for necessary network paths

Question 6

Which statement best describes workload identity?

A. A static password stored in an image

B. A cloud or platform identity assigned to a workload for controlled access

C. A public IP address assigned to a node

D. A registry tag shared by all applications

Correct Answer: B. A cloud or platform identity assigned to a workload for controlled access

Question 7

A function triggered by an object-storage upload should validate the event because:

A. Every storage event is inherently trusted

B. The event or uploaded object may contain malicious or unexpected input

C. Validation disables retries

D. Validation replaces least-privilege permissions

Correct Answer: B. The event or uploaded object may contain malicious or unexpected input

Question 8

Why are centralised logs especially important for ephemeral workloads?

A. Ephemeral workloads always generate more storage

B. Short-lived workloads may disappear before local evidence is collected

C. Centralised logs eliminate the need for monitoring

D. Logs automatically prevent container escapes

Correct Answer: B. Short-lived workloads may disappear before local evidence is collected

Question 9

Which finding should generally receive the highest priority?

A. A low-severity package issue in an isolated workload

B. A documented exception with an expiry date

C. An exposed Kubernetes API allowing unauthorised access

D. A non-production image with an unused library

Correct Answer: C. An exposed Kubernetes API allowing unauthorised access

Question 10

Why does image vulnerability scanning not establish that an image is trustworthy?

A. Scanning cannot identify any package versions

B. A clean scan proves the build system was uncompromised

C. Scanning may miss unknown threats and does not establish build provenance

D. Scanning is relevant only to virtual machines

Correct Answer: C. Scanning may miss unknown threats and does not establish build provenance