9. DevSecOps and Software Supply-Chain Security

Introduction and Context

Continuous integration and continuous deployment have transformed software delivery by making frequent change technically and organizationally feasible. Yet delivery speed also increases the speed at which vulnerabilities, malicious modifications, exposed credentials, and non-compliant components can reach production. Security can no longer be treated as a final inspection performed after development. It must be integrated into the systems, decisions, and automated controls that produce and release software.

DevSecOps extends the collaborative and automated principles of DevOps to security. Its purpose is not simply to add security tools to a pipeline. It is to make security a shared responsibility, introduce security evidence early, automate repeatable controls, and ensure that risks are considered in relation to business impact and delivery context. A secure pipeline must therefore protect both the software product and the process by which that product is built.

The software supply chain includes source code, developers, dependencies, package repositories, build systems, container images, infrastructure definitions, deployment platforms, signing systems, and operational identities. An attacker may target any of these elements. A trustworthy release consequently requires more than testing application behavior; it requires confidence in the origin, integrity, and authorized progression of the software artifact.

This chapter examines threat modeling, secure coding checks, software composition analysis, SAST, DAST, secrets detection, container and infrastructure scanning, artifact signing, provenance, policy-as-code, zero-trust principles, and software supply-chain frameworks. It also considers how organizations should govern these controls without creating unmanageable numbers of false positives or slowing delivery unnecessarily.

Learning Objectives

After studying this chapter, students should be able to:

1. Explain the purpose and organizational implications of DevSecOps.

2. Model threats across application, pipeline, and software supply-chain boundaries.

3. Distinguish SAST, DAST, dependency, secrets, container, and infrastructure scanning.

4. Design security gates that combine risk, evidence, and deployment context.

5. Explain artifact signing, provenance, attestations, SBOMs, and policy-as-code.

6. Apply zero-trust principles to CI/CD identities, runners, repositories, and deployments.

7. Evaluate common software supply-chain frameworks and their limitations.

8. Recommend practical governance, metrics, and improvement strategies for secure delivery.

From DevOps to DevSecOps

DevOps seeks to reduce the separation between development and operations through collaboration, automation, feedback, and shared ownership. DevSecOps applies the same logic to security. Security specialists remain important, but developers, platform engineers, release managers, and operators also become responsible for identifying and reducing security risk.

The central shift is from a security model based primarily on periodic review to one based on continuous assurance. A conventional model may perform a penetration test before release and remediate findings afterward. A DevSecOps model places lightweight controls throughout the lifecycle: threat modeling during design, secure coding guidance during implementation, automated analysis during integration, controlled artifact promotion, and runtime monitoring after deployment.

This shift does not mean that every security decision should be automated or that specialist review is unnecessary. Automated checks are effective for repeatable properties, such as detecting a known vulnerable package or an embedded private key. Human judgment remains necessary for architectural risk, abuse cases, business impact, privacy, and complex attack paths. Effective DevSecOps combines automation with risk-based expertise.

A useful principle is that security controls should be placed as close as possible to the point where a risk is introduced, while evidence should remain available across the delivery chain. A developer should receive a clear explanation when a pull request contains an unsafe construct. A release manager should be able to see which source revision produced an image. An incident responder should be able to identify affected deployments when a dependency vulnerability is announced.

Threat Modeling Across the Delivery Lifecycle

Threat modeling is a structured method for identifying what can go wrong, why it might happen, and how it can be prevented or detected. It is more valuable when performed before implementation because design changes are generally less costly than emergency remediation after deployment.

A practical threat-modeling process begins by defining the system’s purpose, assets, trust boundaries, actors, and data flows. Assets may include customer information, signing keys, source code, deployment credentials, intellectual property, availability, and audit records. Trust boundaries exist where data or authority moves between components—for example, from a public pull request to a build runner or from a CI system to a production cluster.

Threats can be categorized using approaches such as STRIDE: spoofing, tampering, repudiation, information disclosure, denial of service, and elevation of privilege. In supply-chain analysis, additional questions are essential:

• Can an untrusted contributor influence a privileged workflow?
• Can a build process download or execute an unverified dependency?
• Can an attacker replace an artifact after scanning but before deployment?
• Can a compromised runner access long-lived credentials?
• Can a deployment proceed without proof of origin or approval?

Threat modeling should cover three connected surfaces. The application surface includes APIs, authentication, data processing, and business logic. The pipeline surface includes source repositories, runners, build scripts, secrets, plugins, and promotion rules. The operational surface includes registries, clusters, cloud identities, admission controls, and monitoring.

Threat models are not static documents. A change in architecture, dependency source, build platform, identity model, or deployment exposure may invalidate earlier assumptions. Mature teams maintain threat models as living engineering artifacts and connect significant threats to controls, owners, and verification evidence.

Security Controls in the CI/CD Pipeline

Secure coding and developer feedback

Secure coding checks begin with language-specific standards, approved libraries, code review guidance, and secure design patterns. Automated linters and integrated development environment extensions can identify unsafe functions, injection risks, weak cryptographic usage, and insecure configuration before code reaches the central repository. These controls are most useful when findings are precise, actionable, and explained in terms of exploitability and remediation.

Static Application Security Testing

Static Application Security Testing, or SAST, analyzes source code, bytecode, or intermediate representations without executing the application. It can identify patterns associated with injection, insecure deserialization, path traversal, hard-coded credentials, and improper access control. SAST is valuable early in the pipeline because it can run on changed files and provide rapid feedback.

Its limitations must be understood. SAST may report code that is technically suspicious but unreachable, or miss vulnerabilities that depend on runtime configuration, external services, or business logic. It can also create alert fatigue if every finding is treated as an automatic release blocker. Teams should classify findings by confidence and severity, establish remediation expectations, and permit documented exceptions with expiry dates.

Software composition analysis and dependency security

Modern applications rely heavily on open-source and commercial components. Software Composition Analysis, or SCA, inventories direct and transitive dependencies, compares versions with vulnerability databases, identifies licenses, and may detect components that are abandoned or outside organizational policy.

Dependency risk is not determined by a vulnerability score alone. Teams should consider whether the vulnerable function is reachable, whether a fix exists, whether the component is exposed to an attacker, and whether compensating controls are present. Lockfiles, private mirrors, version pinning, dependency review, and automated update mechanisms improve reproducibility and reduce the risk of silently changing inputs.

Secrets detection

Secrets include passwords, API tokens, private keys, signing credentials, cloud access keys, and database connection strings. Secrets detection tools inspect commits, pull requests, file systems, container layers, and sometimes build logs. Prevention is preferable to discovery: credentials should be stored in dedicated secret managers, injected at runtime when possible, and scoped to the minimum required permissions.

If a secret is committed, deleting the line is insufficient because repository history, caches, logs, or artifacts may retain it. The credential must be revoked or rotated, affected systems assessed, and the event investigated. Secret scanning should therefore be connected to an operational response process rather than treated as a simple pass/fail check.

Dynamic and interactive testing

Dynamic Application Security Testing, or DAST, tests a running application from an external perspective. It can detect issues such as insecure headers, authentication weaknesses, exposed endpoints, and certain injection vulnerabilities. DAST is particularly useful in an integration or staging environment where realistic configuration and network behavior are present.

Interactive Application Security Testing combines runtime observation with code context. It can help distinguish exploitable paths from theoretical warnings, although it introduces instrumentation and operational complexity. DAST generally cannot prove the absence of vulnerabilities; authentication coverage, test data, endpoint discovery, and environment similarity strongly affect results.

Container and infrastructure scanning

Container scanning examines image layers, operating-system packages, application dependencies, configuration, and sometimes malware indicators. A secure container process uses minimal base images, removes unnecessary tools, runs as a non-root user where practical, and rebuilds images when base components require updates. Images should be scanned before publication and again according to an established vulnerability-management policy.

Infrastructure-as-code scanning reviews templates for insecure network exposure, excessive permissions, unencrypted storage, weak identity configuration, and unsafe defaults. It should be complemented by runtime configuration assessment because a secure template may be altered by manual actions or platform behavior. Scanning is evidence, not a guarantee: a clean result depends on the quality and currency of rules, databases, and configuration assumptions.

Artifact Integrity, Provenance, and Release Trust

A pipeline must establish not only that an artifact passed tests, but also where it came from and whether it was changed. Artifact signing uses cryptographic keys to bind an artifact or digest to an identity. Verification at deployment can ensure that only artifacts signed by an approved builder or release authority are admitted.

Signing is meaningful only when key management is trustworthy. Private keys require controlled storage, rotation, access logging, and recovery procedures. A signature does not certify that software is secure; it certifies a relationship between an artifact and a signing identity. Policy must therefore specify who may sign, what was verified, and where signed artifacts may be used.

Provenance records describe how an artifact was produced. Typical information includes the source repository and revision, builder identity, build parameters, dependencies, timestamps, and results of relevant checks. An attestation is a verifiable statement about an artifact, such as “this image was built from a specified commit using an approved workflow.” Provenance supports incident response, compliance, and deployment decisions.

A Software Bill of Materials, or SBOM, is an inventory of components and their relationships in a software product. Formats such as SPDX and CycloneDX enable exchange between tools, although formats alone do not ensure accuracy. SBOMs are most useful when generated automatically from authoritative build inputs, associated with immutable artifact identifiers, and refreshed when software changes.

Policy-as-Code and Security Gates

Policy-as-code expresses organizational rules in machine-readable form. Examples include: production images must be signed; deployments may not use critical unaddressed vulnerabilities; containers must not run with privileged access; infrastructure must encrypt sensitive storage; and only approved registries may be used.

Policies can be enforced at several points: pull requests, build workflows, artifact registries, deployment controllers, and cluster admission. Early enforcement gives developers rapid feedback, while later enforcement protects against bypasses or changes after the initial scan. Effective designs use layered controls rather than relying on a single gate.

A security gate should be proportionate to risk. Blocking every medium-severity finding may cause teams to circumvent controls, whereas allowing an unsigned production artifact creates a direct integrity failure. Organizations should define severity thresholds, exploitability considerations, exception ownership, expiration dates, and emergency procedures. A useful exception is explicit, justified, time-limited, and visible; it is not a permanent suppression hidden in a tool configuration.

Zero Trust in CI/CD

Zero trust rejects the assumption that network location, prior access, or organizational membership automatically establishes trust. In CI/CD, each request should be authenticated, authorized, and evaluated in context. Important principles include least privilege, short-lived credentials, workload identity, separation of duties, continuous verification, and strong auditability.

Build runners deserve special attention. A compromised runner may access source code, secrets, package credentials, and signing systems. Organizations should isolate workloads, prefer ephemeral runners for sensitive jobs, restrict network egress, pin or verify third-party actions, and prevent untrusted code from executing with production privileges. Pull requests from external contributors should not automatically receive access to privileged secrets.

Deployment identities should be distinct from build identities where possible. A build system may create and sign an image, while a deployment controller verifies the signature and applies policy. This separation limits the consequences of a compromise and creates clearer evidence about which identity performed each action.

Frameworks for Supply-Chain Security

The NIST Secure Software Development Framework organizes secure development practices around preparing the organization, protecting software, producing well-secured software, and responding to vulnerabilities. It is useful as a governance and process reference because it connects technical controls with organizational responsibilities.

Supply-chain Levels for Software Artifacts, commonly known as SLSA, focuses on increasing confidence in artifact provenance and build integrity through graduated requirements. Its concepts include authenticated source, controlled builds, provenance, and stronger protection against tampering. SLSA is especially relevant when an organization needs to demonstrate how a release was produced.

The Open Source Security Foundation’s Scorecard project provides checks for selected open-source project practices, such as branch protection and dependency updating. It can inform component selection but should not be treated as a complete security judgment. The National Institute of Standards and Technology Cybersecurity Framework can also provide a broader risk-management structure, while software security maturity models can help organizations assess capability progression.

Frameworks should guide decisions rather than become compliance exercises. A company may satisfy a documentation requirement while leaving signing keys exposed or granting excessive runner permissions. The meaningful question is whether the framework’s practices reduce realistic threats and produce reliable evidence.

An Illustrative Case: Securing a Financial Services Delivery Chain

Background

A fictional digital payments company, PayLink, operates a microservices platform deployed to a managed container environment. It releases several times per day. The engineering organization uses multiple open-source libraries, container images from public registries, and a shared CI runner pool.

Situation and problem

A security review discovers that a build script downloads an unpinned tool at runtime, several repositories contain long-lived cloud credentials, and production deployments are based on mutable image tags. The company also cannot reliably determine which source revision produced a deployed image. Developers complain that existing vulnerability scans generate hundreds of warnings and frequently block routine releases.

Analysis

Threat modeling identifies four high-priority risks: malicious modification of a build input, compromise of a shared runner, unauthorized use of cloud credentials, and deployment of an altered image after testing. The organization distinguishes these integrity risks from lower-confidence dependency findings. It also recognizes that the absence of provenance will make incident response slow and uncertain.

Strategy and solution

PayLink first replaces long-lived cloud credentials with workload identities and restricts permissions by repository and environment. External pull requests run in isolated jobs without access to privileged secrets. Build tools and dependencies are pinned by version and checksum, and approved packages are retrieved through a controlled repository mirror.

The company introduces SAST, secret scanning, SCA, container scanning, and infrastructure-as-code checks. High-confidence secrets, exploitable critical vulnerabilities, and privileged container configurations block merges. Lower-confidence findings create tracked work items rather than automatic failures. Exceptions require an owner, rationale, compensating control, and expiry date.

The build system generates an SBOM and provenance attestation, signs the image digest using a protected signing service, and stores the evidence with the artifact. The deployment controller admits only signed images whose provenance identifies an approved workflow and source repository. Production references immutable digests rather than mutable tags. A dashboard reports vulnerability age, exception age, signed-artifact coverage, and time from disclosure to remediation.

Outcome

The company gains the ability to trace production artifacts to source revisions and build workflows. Credential exposure is reduced, and deployment of unverified images is prevented. Developers initially experience workflow changes, but targeted findings and documented policy thresholds reduce alert fatigue. The organization does not eliminate all vulnerabilities; instead, it improves the reliability of risk prioritization and the speed of response.

Learning points

The case illustrates that supply-chain security is a system of controls rather than a collection of scanners. Identity, reproducible inputs, artifact integrity, evidence, and proportionate policy must work together. It also demonstrates that governance quality matters: poorly tuned blocking rules can encourage bypass behavior, while weak release controls can leave serious integrity risks unaddressed.

Challenges, Limitations, and Common Mistakes

A frequent mistake is equating a successful scan with secure software. Scanners have incomplete rule sets, false positives, false negatives, and limited understanding of business logic. Another mistake is adding controls without assigning ownership for triage and remediation. Unowned findings accumulate and weaken confidence in the program.

Organizations may also over-centralize security, requiring a specialist team to approve every change. This creates bottlenecks and discourages responsibility in product teams. Conversely, completely delegating security to developers without training, platform support, or specialist review produces inconsistent results.

Other risks include untrusted third-party pipeline actions, excessive permissions, mutable tags, unprotected branches, insecure build caches, leaked logs, unreviewed policy changes, and inadequate recovery procedures for signing keys. A mature program tests its own controls, including whether a compromised runner can access unrelated environments and whether a revoked signer is actually rejected.

Security controls can affect developer experience, build duration, privacy, and cost. These trade-offs should be measured and managed. Incremental scanning, parallel execution, caching of verified dependencies, risk-based gates, and feedback within pull requests can preserve delivery flow without abandoning assurance.

Governance, Ethics, and Regulatory Considerations

Software supply-chain governance involves accountability, traceability, and responsible disclosure. Organizations should define who owns vulnerabilities, who approves exceptions, who controls production signing, and how evidence is retained. Logs and provenance may contain sensitive information, so access and retention must comply with privacy and legal requirements.

Software licenses are also part of supply-chain governance. SCA programs should identify obligations and restrictions, but legal interpretation may require specialist review. When vulnerabilities affect customers, organizations face ethical decisions about disclosure timing, notification, and mitigation. Concealing known material risk may protect short-term reputation while increasing harm to users and partners.

Regulatory environments increasingly expect secure development practices, vulnerability management, access control, and demonstrable software integrity. Compliance should be treated as a minimum obligation. A control that exists only to produce an audit document but is not operationally effective provides limited protection.

Current Developments and Emerging Directions

Provenance-aware deployment is becoming more practical as build platforms and admission controllers support attestations and signature verification. SBOM exchange is also expanding, particularly for vulnerability response and supplier assurance. Organizations are moving from static vulnerability lists toward reachability analysis, exploit intelligence, and prioritization based on actual exposure.

Keyless or short-lived signing approaches can reduce the risks associated with permanently stored private keys, although they create new requirements for identity verification, trust-root management, and availability. Automated dependency updates, reproducible builds, hermetic build environments, and isolated build services are receiving increased attention.

Artificial intelligence can assist with code review, vulnerability explanation, test generation, and remediation suggestions. However, AI-generated code may reproduce insecure patterns, introduce licensing uncertainty, or create false confidence. AI tools must be subject to the same review, dependency, secrets, and provenance controls as other development inputs.

Best Practices for Implementation

A practical implementation sequence is to inventory software and delivery assets, identify high-impact threats, establish ownership, and protect identities before adding numerous scanners. Organizations should then introduce high-value checks with clear remediation guidance, generate SBOMs and provenance, sign artifacts, and enforce deployment verification.

The following principles provide a concise operating standard:

• Treat source, build systems, dependencies, artifacts, and deployment identities as security assets.
• Prefer short-lived, least-privileged credentials and isolated build environments.
• Pin and verify important inputs; avoid unreviewed downloads during builds.
• Scan early, but validate findings in their runtime and business context.
• Use immutable artifact references and verify signatures before deployment.
• Make provenance and SBOM evidence automatically available.
• Express repeatable rules as policy-as-code and test policy changes.
• Make exceptions visible, justified, owned, and time-limited.
• Measure remediation and assurance outcomes, not merely the number of scans.
• Continuously test whether pipeline controls withstand realistic compromise scenarios.

Chapter Summary

DevSecOps integrates security with the collaborative, automated lifecycle of DevOps. Its scope includes application code, dependencies, build infrastructure, artifacts, identities, deployment platforms, and operational evidence. Threat modeling identifies realistic attack paths and connects them to controls. SAST, DAST, SCA, secrets detection, container scanning, and infrastructure scanning provide complementary forms of assurance, but none is complete in isolation.

Artifact signing establishes integrity relationships, while provenance and SBOMs provide evidence about origin and composition. Policy-as-code turns security expectations into repeatable controls, and zero-trust principles require continuous authentication, authorization, least privilege, and separation of duties throughout CI/CD. Frameworks such as NIST SSDF and SLSA help structure organizational improvement, but their value depends on effective implementation rather than documentation alone. The central professional challenge is to create controls that are strong enough to reduce material risk and usable enough to remain part of everyday delivery.

Key Terms

Threat modeling: A structured process for identifying assets, threats, vulnerabilities, and mitigations.

DevSecOps: The integration of security responsibilities and controls into DevOps practices and pipelines.

SAST: Static Application Security Testing that analyzes code without executing the application.

DAST: Dynamic Application Security Testing that examines a running application for security weaknesses.

SCA: Software Composition Analysis of dependencies, licenses, versions, and known vulnerabilities.

SBOM: A machine-readable inventory of software components and their relationships.

Secret: Sensitive authentication or cryptographic material such as a token, password, or private key.

Container scanning: Analysis of container images for vulnerabilities, malware, and insecure configuration.

Infrastructure-as-code scanning: Automated examination of infrastructure templates for insecure settings.

Artifact: A build output, such as a binary, package, or container image.

Artifact signing: Use of cryptography to associate an artifact with an authorized identity.

Provenance: Evidence describing the source, process, inputs, and builder that produced an artifact.

Attestation: A verifiable statement about an artifact or process.

Policy-as-code: Machine-readable rules used to evaluate or enforce organizational policies.

Zero trust: A security approach that requires continuous verification rather than implicit trust.

Supply chain: The connected people, processes, systems, and components involved in producing software.

SLSA: A framework for improving software artifact provenance and build integrity.

Exception: An approved, documented, and usually time-limited deviation from a security policy.

Least privilege: Granting an identity only the permissions necessary for its authorized task.

Reproducible build: A build process capable of producing equivalent outputs from the same defined inputs.

Review Questions

Short Answer Questions

1. Why is security treated as a shared responsibility in DevSecOps?

2. What information does provenance provide about a software artifact?

3. Distinguish SAST from DAST.

4. Why is rotating a committed secret necessary even after the secret is deleted from a file?

5. What is the purpose of an SBOM?

Long Answer Questions

1. Explain how threat modeling can be applied to application, pipeline, and operational surfaces.

2. Discuss the complementary strengths and limitations of SAST, DAST, SCA, secrets detection, and container scanning.

3. Explain how signing, provenance, attestations, and immutable artifact references contribute to release trust.

4. Analyze the role of zero-trust principles in protecting CI/CD runners, credentials, and deployment systems.

5. Compare the practical contributions of NIST SSDF and SLSA to software supply-chain security.

Analytical / Application Questions

1. A team has a large number of medium-severity dependency findings and begins ignoring all scanner results. Design a risk-based triage and policy approach that improves security without creating excessive release disruption.

2. An organization allows external pull requests to execute on shared runners with access to cloud credentials. Identify the major threats and propose a safer pipeline architecture.

3. A production image was found to contain a vulnerable library, but the organization cannot identify its source commit or build process. Explain which supply-chain controls were missing and recommend a remediation plan.

22. Multiple Choice Questions with Answers

Question 1

Which statement best describes the primary purpose of DevSecOps?

A. To transfer all security responsibilities to a central security team

B. To integrate security responsibilities and automated controls throughout software delivery

C. To replace application testing with vulnerability scanning

D. To prevent developers from deploying software without manual approval

Correct Answer: B. To integrate security responsibilities and automated controls throughout software delivery

Question 2

Which activity is most appropriate for identifying threats at trust boundaries in a CI/CD system?

A. Threat modeling

B. Container image compression

C. Database indexing

D. Load testing only

Correct Answer: A. Threat modeling

Question 3

What is a principal limitation of SAST?

A. It cannot analyze source code before execution

B. It is unable to detect any coding weaknesses

C. It may report unreachable code and miss runtime or business-logic weaknesses

D. It can only operate against container images

Correct Answer: C. It may report unreachable code and miss runtime or business-logic weaknesses

Question 4

A container image passes a vulnerability scan but is deployed using a mutable tag that another process can replace. Which control most directly addresses this integrity risk?

A. Using an immutable image digest and verifying an artifact signature

B. Increasing the number of application unit tests

C. Adding more comments to the Dockerfile

D. Running the same scan only after deployment

Correct Answer: A. Using an immutable image digest and verifying an artifact signature

Question 5

What is the main purpose of an SBOM?

A. To encrypt all source code in a repository

B. To inventory software components and their relationships

C. To replace artifact signatures

D. To grant deployment permissions to developers

Correct Answer: B. To inventory software components and their relationships

Question 6

Which practice best reflects zero-trust principles in a CI/CD pipeline?

A. Trusting every job that runs inside the corporate network

B. Giving all build runners permanent administrator privileges

C. Using short-lived, least-privileged identities and continuously verifying requests

D. Allowing production credentials in public pull-request workflows

Correct Answer: C. Using short-lived, least-privileged identities and continuously verifying requests

Question 7

A provenance attestation for an image should primarily help an organization determine which fact?

A. Whether the image has attractive user-interface design

B. Which source, builder, inputs, and process produced the image

C. Whether every possible runtime vulnerability has been eliminated

D. How many users will deploy the image

Correct Answer: B. Which source, builder, inputs, and process produced the image

Question 8

Which is the strongest example of a well-designed security exception?

A. A permanent suppression with no owner

B. An undocumented verbal approval

C. A visible exception with justification, compensating controls, an owner, and an expiry date

D. A global setting that disables the failing scanner

Correct Answer: C. A visible exception with justification, compensating controls, an owner, and an expiry date

Question 9

Why should a secret committed to repository history normally be revoked or rotated?

A. Because deleting the current line guarantees the repository is corrupted

B. Because historical commits, logs, or caches may still contain the credential

C. Because secret scanners cannot detect newly created secrets

D. Because all source repositories automatically publish credentials

Correct Answer: B. Because historical commits, logs, or caches may still contain the credential

Question 10

Which outcome best demonstrates effective software supply-chain security?

A. The organization runs a scanner on every commit but cannot trace deployments

B. The organization blocks every warning regardless of exploitability

C. The organization can verify artifact origin, enforce deployment policy, and respond rapidly to component risk

D. The organization performs one annual penetration test and stores the report

Correct Answer: C. The organization can verify artifact origin, enforce deployment policy, and respond rapidly to component risk