Artifact, Package, and Dependency Management

Introduction

Continuous integration and continuous delivery (CI/CD) do not end when source code is compiled or tested. A successful pipeline produces outputs that must be identified, stored, secured, retrieved, promoted, and eventually retired. These outputs may include executable binaries, libraries, operating-system packages, container images, deployment charts, documentation, and metadata such as software bills of materials (SBOMs) and provenance statements.

Artifact management provides the discipline for handling these outputs throughout their lifecycle. Package management focuses on distributing reusable software components and resolving their dependencies. Together, they create a controlled link between source code, build processes, and deployable systems.

Poor artifact and dependency management creates serious risks. A deployment may use a different binary from the one tested; a mutable container tag may silently point to new content; an obsolete vulnerable library may remain in production; or a build may become impossible to reproduce because its dependencies have disappeared. Effective DevOps practice therefore treats build outputs as versioned, traceable, and governed products rather than temporary files.

Learning Objectives

By the end of this chapter, you should be able to:

• distinguish artifacts, packages, dependencies, repositories, and registries;
• explain versioning and apply semantic versioning appropriately;
• design practices for immutable and traceable build outputs;
• describe package and container registry workflows;
• manage direct, transitive, runtime, and development dependencies;
• apply provenance, SBOM, vulnerability, retention, and access controls;
• analyze practical trade-offs in artifact-management design.

1. Core Concepts and Terminology

Artifacts

An artifact is a file or collection of files produced, consumed, or exchanged by a software-delivery process. Examples include a compiled Java archive, a Python wheel, a Debian package, a JavaScript bundle, a Terraform module, a Helm chart, and a container image. An artifact can be deployable, such as an executable, or supporting, such as a test report or SBOM.

A build artifact should have a clear identity. Its identity normally includes:

1. a name;
2. a version or immutable content identifier;
3. a format and platform, when relevant;
4. metadata describing its origin and contents.

Packages

A package is a distributable unit that normally includes software and metadata describing how it is installed or consumed. A package manager can locate packages, resolve dependencies, verify integrity, and install compatible versions. Examples include npm packages, Maven artifacts, NuGet packages, Python packages, Debian packages, and RPM packages.

The terms artifact and package overlap, but they are not identical. An artifact is a broad term for a build output. A package is usually an artifact prepared for distribution through a package ecosystem, with a package manifest and dependency information.

Repositories and registries

A repository stores software artifacts and their metadata. A registry is a repository service organized around particular package or image formats. A registry may be public, private, hosted by an organization, or operated as a proxy cache for external dependencies.

A typical organization uses several repositories:

A registry is not merely a file server. It may enforce authentication, repository permissions, malware scanning, retention rules, replication, and audit logging.

Dependencies

A dependency is a component required by software to build, test, run, or operate. A direct dependency is declared by the application. A transitive dependency is introduced by another dependency. For example, an application may directly depend on a web framework, while the framework introduces logging and serialization libraries.

Dependencies can also be classified by lifecycle:

• Build dependency: required to compile or package software.
• Test dependency: required to execute automated tests.
• Runtime dependency: required when the application executes.
• Development dependency: useful to developers but not needed in production.
• Platform dependency: supplied by an operating system, runtime, database, or cloud service.

This classification supports smaller production images, faster deployments, and more accurate vulnerability assessment.

2. The Artifact Lifecycle

An artifact typically passes through the following lifecycle:

1. Build: source code and dependencies are transformed into an output.
2. Validate: tests, quality checks, security scans, and policy checks are performed.
3. Publish: the artifact is uploaded to a controlled registry.
4. Promote: the same artifact is approved for progressively more trusted environments.
5. Deploy: the artifact is installed or executed.
6. Monitor: operational behavior and vulnerabilities are observed.
7. Retain or retire: the artifact is preserved for a justified period or removed according to policy.

A central principle is build once, promote many times. A tested artifact should move from development to testing and production without being rebuilt for each environment. Environment-specific configuration should be supplied separately, where possible. Rebuilding can change timestamps, dependency resolution, compiler behavior, or generated content, thereby breaking equivalence between tested and deployed outputs.

Versioned versus immutable outputs

Versioning gives an artifact a human-readable identity, while immutability ensures that the content associated with an identity cannot change. These properties are complementary.

For example, payments-api:2.4.1 is a readable image tag. If the registry permits that tag to be overwritten, it is not an immutable reference. A digest such as sha256:... identifies content by cryptographic hash and is suitable for deployment pinning. A robust process may display the semantic version to people while recording and deploying the digest.

Immutability does not mean that artifacts can never be deleted. It means that an existing version or digest cannot be silently replaced with different content. Deletion, retention, and archival are separate governance decisions.

3. Semantic Versioning and Release Identity

Semantic Versioning, commonly written as SemVer, provides a convention for communicating compatibility through a version number of the form MAJOR.MINOR.PATCH.

• MAJOR increases when incompatible public API changes are introduced.
• MINOR increases when backward-compatible functionality is added.
• PATCH increases when backward-compatible defects are corrected.

For example, changing 1.4.2 to 1.4.3 suggests a compatible fix. Adding a compatible feature suggests 1.5.0. Removing or changing an API in an incompatible way suggests 2.0.0.

SemVer is a communication convention, not a guarantee. It works only when a project defines its public interface and follows compatibility discipline. A project that makes undocumented database changes or breaks configuration behavior may introduce a major operational change even if its version number changes only from 1.4.2 to 1.4.3.

Pre-release identifiers such as 2.0.0-rc.1 communicate that a version is not yet a final release. Build metadata may record additional information without determining precedence, for example 1.8.0+build.542.

Version ranges and reproducibility

Dependency declarations often use ranges. A declaration such as ^3.2 may permit compatible updates, while an exact declaration such as 3.2.7 permits only one version. Ranges improve access to fixes but can reduce reproducibility if resolution changes between builds.

A practical compromise is:

• declare acceptable ranges in the project manifest;
• generate and commit a lockfile containing the exact resolved versions;
• use the lockfile in CI and production builds;
• update dependencies through controlled, reviewable changes.

A lockfile is not a substitute for security review. It records a choice; it does not prove that the chosen component is safe.

4. Package and Container Registry Workflows

Publishing packages

A package-publishing workflow normally validates the package name and version, runs tests, creates the package, signs or attests to it where required, and uploads it to a registry. The registry should reject accidental overwrites of released versions. Publication credentials should be short-lived and scoped to the target repository.

Consumers should normally use an organizational proxy or approved repository rather than retrieving arbitrary packages directly from the public internet. A proxy improves availability, reduces repeated downloads, and enables organizational policy, but cached content must still be scanned and governed.

Container images

A container image is composed of filesystem layers and configuration metadata. Registries commonly store images using the Open Container Initiative (OCI) image format. Tags such as latest, stable, or production are convenient labels, but they are mutable unless explicitly protected.

Recommended practice includes:

• build from a trusted and maintained base image;
• use a small runtime image when practical;
• scan the image and its operating-system packages;
• run as a non-root user where feasible;
• push the image once and promote it by digest;
• record the source revision, builder, base image, and build time;
• avoid embedding passwords, tokens, or private keys in layers;
• apply repository permissions and signing or attestation controls.

A container digest identifies the image manifest and its referenced content. Deployment systems should record the digest actually used, not merely the tag requested by a user.

Promotion models

Two common promotion models are:

1. Copy promotion: an artifact is copied from one repository to another. This can provide separation but may create duplicate storage and requires proof that the copy is identical.
2. Reference promotion: an approval record points to the original immutable artifact. This avoids duplication but requires reliable access and governance.

In either model, promotion should be an authorization event, not a rebuild. A release record should connect the artifact to its source commit, pipeline execution, test evidence, approvers, and deployment environments.

5. Provenance, Integrity, and Supply-Chain Security

Integrity

Integrity means confidence that an artifact has not been altered unexpectedly. Transport encryption protects data in transit, while checksums and cryptographic signatures support content verification. A checksum can detect accidental or unauthorized modification when the expected digest is obtained through a trusted channel. A digital signature additionally associates the content with a signing identity, subject to the security of the signing key and verification process.

Signing does not make malicious software safe. It establishes an assertion about origin or approval. Organizations must protect signing keys, rotate them appropriately, limit signing authority, and retain verification evidence.

Provenance

Provenance describes how an artifact was produced. Useful provenance includes:

• source repository and commit identifier;
• build definition and pipeline run;
• builder identity and environment;
• input dependencies and base images;
• timestamps and target platform;
• test and scan results;
• signing or attestation information.

Frameworks such as SLSA provide a vocabulary and maturity model for improving software supply-chain assurance. The precise level of assurance depends on the build platform, isolation, identity controls, and verifiability of the generated provenance.

Software bills of materials

An SBOM is an inventory of components in a software product. Common formats include SPDX and CycloneDX. An SBOM supports vulnerability response, license review, incident investigation, and customer disclosure. It should be generated from reliable build information and associated with the exact artifact version or digest.

SBOMs have limitations. They may omit dynamically downloaded components, inaccurately identify vendored code, or become stale when the artifact changes. They therefore complement, rather than replace, scanning and engineering review.

6. Dependency Governance

Dependency management is both a technical and organizational activity. A useful policy should address selection, approval, updating, monitoring, and removal.

Selection and approval

Teams should prefer actively maintained components with clear licensing, suitable security practices, and a healthy release process. Popularity alone is not sufficient evidence of quality. Important questions include:

• Is the component maintained and supported?
• Does it have a transparent vulnerability-reporting process?
• Is its license compatible with the product?
• Does it introduce unnecessary functionality or transitive dependencies?
• Can the organization obtain and verify its source and provenance?

Pinning and updating

Pinning exact versions improves repeatability. However, permanently pinning old versions creates security and maintenance risk. A mature process combines reproducible builds with scheduled update work, automated compatibility testing, and controlled exceptions.

Automated tools can open update pull requests, identify vulnerable versions, and propose remediation. Automation should not blindly merge updates into critical systems. Risk-based review remains necessary, especially for major versions, authentication libraries, parsers, cryptographic components, and network-facing packages.

Dependency confusion and substitution

Dependency confusion occurs when a build retrieves a malicious public package instead of an intended private package with the same name. Controls include private namespace conventions, explicit repository configuration, proxy allowlists, package-source restrictions, and monitoring of unexpected downloads.

Typosquatting uses names that resemble legitimate packages. Developers and build systems should verify package identity, use approved sources, and review new dependencies. A package-locking mechanism reduces resolution changes but cannot prevent a deliberately selected malicious package.

Vulnerability response

A vulnerability process should identify affected artifacts, determine exploitability, prioritize remediation, rebuild affected outputs, and verify deployment. Severity scores help triage but do not replace context. A vulnerable library may be unreachable or disabled in one product and directly exposed in another.

The organization should maintain an inventory mapping deployed artifacts to their components. Without this relationship, a newly disclosed vulnerability can require a slow, manual search across systems.

7. Retention, Storage, and Access Controls

Retention policy balances cost, operational recovery, audit needs, and legal obligations. It should define which artifacts are retained, for how long, and under what deletion or archival conditions.

A useful policy may distinguish:

Deletion must not remove the only copy needed for a supported release, regulatory evidence, or incident investigation. Replication and backup strategies should consider registry metadata as well as binary layers.

Access should follow least privilege. Developers may publish to development repositories, while release automation may promote artifacts and production systems may pull without publishing. Administrative actions such as deletion, permission changes, and key rotation should be logged and reviewed.

8. Illustrative Case: Northstar Payments

Background

Northstar Payments is an illustrative mid-sized organization operating a Java service and a containerized reporting service. Its teams build successfully, but production incidents reveal that the image tagged release sometimes differs from the image tested in staging. A critical vulnerability announcement also requires several days of manual investigation because the organization does not know which services contain the affected library.

Problem analysis

The pipeline rebuilds the application separately for each environment. Dependency versions are expressed as ranges without consistently committed lockfiles. Developers pull packages directly from public repositories. Container tags are mutable, and the registry has no reliable link between an image and its source commit. Old images are retained indefinitely, increasing storage costs while still failing to provide useful release evidence.

Approach

Northstar introduces the following controls:

1. Each build creates a versioned package and an image once.
2. A lockfile records resolved dependency versions.
3. Images are pushed with a release tag and identified in deployment manifests by digest.
4. Released versions cannot be overwritten.
5. A private proxy caches approved external packages and applies source policies.
6. The pipeline generates an SBOM and provenance statement for every release.
7. Security scanning occurs before promotion, with documented risk-based exceptions.
8. Promotion records connect source commit, pipeline run, artifact digest, test evidence, and approver.
9. Production and supported rollback versions are retained; temporary snapshots expire automatically.
10. A scheduled dependency-update process creates reviewed pull requests.

Outcome

When a vulnerability is disclosed, Northstar queries its artifact inventory and identifies affected services by package and version. The deployment platform confirms the exact image digest in each environment. The team rebuilds only affected applications, verifies tests and scans, and promotes the new immutable outputs without rebuilding between environments.

Storage costs fall after snapshot cleanup, while release support improves because historical production artifacts remain available. The organization still faces trade-offs: proxy administration requires effort, signing infrastructure needs protection, and teams must understand why a human-readable tag is not enough for deployment identity.

Learning points

The case illustrates that artifact management is a system of controls rather than a single registry product. Reproducibility, immutability, provenance, dependency visibility, and retention reinforce one another. Removing any one of them weakens the evidence chain between source and production.

9. Practical Implementation Pattern

A certificate-level implementation can begin with the following sequence:

1. Inventory all artifact types and their consumers.
2. Select repositories for internal packages, containers, and approved external dependencies.
3. Define naming and versioning conventions.
4. Protect released versions from overwrite.
5. Generate lockfiles and make CI use them.
6. Publish build outputs only after tests and policy checks pass.
7. Generate SBOMs and provenance as pipeline outputs.
8. Promote immutable artifacts rather than rebuilding them.
9. Pin production deployments to digests or exact package versions.
10. Monitor vulnerabilities, stale dependencies, registry access, and retention results.

Teams should document exceptions. For example, a rapidly changing development image may be mutable in a sandbox, but it must not be confused with a production release. Exceptions should have an owner, expiration date, and compensating controls.

10. Challenges and Limitations

Immutability versus operational flexibility

Immutable artifacts improve confidence but can make emergency correction less convenient. The solution is not to overwrite production artifacts; it is to create and promote a new corrective version. Emergency procedures should be fast without abandoning traceability.

Availability and registry dependency

A pipeline or cluster that depends on a registry can fail when the registry is unavailable. Caching, replication, backup, and disaster-recovery exercises reduce this risk. Teams should understand whether deployment requires access to every image layer or package at deployment time.

False positives and scanning limits

Vulnerability scanners may report issues that are not exploitable, while failing to detect novel or application-specific weaknesses. Scan results should support prioritization, not replace secure design, testing, patching, and human judgment.

License and privacy concerns

Dependency metadata may reveal product architecture or internal package names. Access controls should protect private repositories and build logs. License obligations may require notices, source availability, or other actions; these obligations should be reviewed before distribution.

AI-assisted dependency and code generation

AI tools can suggest packages, code, and configuration. They may also recommend abandoned, vulnerable, or incorrectly licensed components. Generated dependency changes require the same source verification, testing, license review, and security controls as human-authored changes. AI can assist inventory and update analysis, but accountability remains with the engineering organization.

Best Practices Checklist

• Use a registry appropriate to each package and artifact format.
• Give every release a unique, meaningful identity.
• Prevent overwriting of released versions.
• Use tags for human communication and digests or exact versions for deployment.
• Build once and promote the same artifact.
• Commit lockfiles and update dependencies deliberately.
• Separate development, test, and production dependencies.
• Generate SBOM and provenance information.
• Scan artifacts and assess exploitability in context.
• Restrict publishing, promotion, deletion, and administration privileges.
• Define retention, backup, archival, and recovery policies.
• Preserve evidence linking source, build, artifact, approval, and deployment.

Chapter Summary

Artifact, package, and dependency management connects software source code to trustworthy deployment outputs. Artifacts should be identifiable, versioned, and protected from silent replacement. Semantic versioning communicates intended compatibility, while lockfiles and exact references improve reproducibility. Registries provide controlled storage, distribution, scanning, access management, and audit capabilities.

The strongest delivery process builds an artifact once, validates it, records provenance and an SBOM, and promotes the same immutable content through environments. Dependency governance must address direct and transitive components, licensing, vulnerability response, package-source risks, and controlled updates. Retention and access policies preserve operational and audit value without allowing repositories to become unmanaged archives. These practices collectively improve reliability, security, recovery, and confidence in the software supply chain.

Key Terms

Artifact: A file or collection of files produced or consumed by a software process.

Package: A distributable software unit containing code and metadata.

Dependency: A component required to build, test, run, or operate software.

Direct dependency: A dependency explicitly declared by an application.

Transitive dependency: A dependency introduced through another dependency.

Package manager: A tool that resolves, installs, updates, and verifies packages.

Registry: A service that stores and distributes packages or container images.

Repository: A controlled location for artifacts and associated metadata.

Semantic Versioning: A versioning convention using MAJOR, MINOR, and PATCH components.

Lockfile: A file recording exact dependency versions selected for a build.

Immutability: The property that released content cannot be silently replaced.

Digest: A content-derived identifier, commonly a cryptographic hash.

Container image: A packaged filesystem and configuration used to run a container.

Promotion: Authorization to move or approve an artifact for a more trusted environment.

Provenance: Evidence describing how, where, and from what inputs an artifact was built.

SBOM: A software bill of materials listing components in a software product.

Retention policy: Rules governing how long artifacts and evidence are stored.

Dependency confusion: A substitution attack involving private and public package names.

Review Questions

Short Answer Questions

1. What is the difference between an artifact and a package?
2. Why is a container tag such as latest insufficient as a production identity?
3. What do the MAJOR, MINOR, and PATCH components represent in Semantic Versioning?
4. What information can an SBOM provide?
5. Why should released artifacts be protected from overwriting?

Long Answer Questions

1. Explain the artifact lifecycle from build through retirement, including the purpose of promotion.
2. Describe how lockfiles, version ranges, and scheduled updates can be used together.
3. Explain the role of provenance, signatures, and digests in software supply-chain security.
4. Compare hosted, proxy, and virtual repositories and identify a suitable use for each.
5. Discuss the main elements of an effective registry retention and access-control policy.

Analytical / Application Questions

1. A team deploys an image by the tag production, and the tag is overwritten after staging tests complete. Analyze the risks and propose a corrected workflow.
2. A newly disclosed vulnerability affects a popular transitive library. Design the steps an organization should take to identify affected deployments, remediate them, and preserve evidence.
3. An organization wants to permit flexible dependency ranges to receive fixes quickly but also requires reproducible production builds. Recommend a dependency-management policy and justify it.

Multiple Choice Questions with Answers

Question 1

Which statement best describes an immutable artifact?

A. It is stored only on a developer workstation.

B. Its released content cannot be silently replaced.

C. It always contains no dependencies.

D. It must be rebuilt for every environment.

Correct Answer: B. Its released content cannot be silently replaced.

Question 2

In Semantic Versioning, which change normally indicates a backward-compatible feature addition?

A. Increasing the PATCH component

B. Changing only build metadata

C. Increasing the MINOR component

D. Removing the version number

Correct Answer: C. Increasing the MINOR component

Question 3

What is the primary purpose of a lockfile?

A. To record exact dependency versions resolved for a build

B. To encrypt all source code

C. To replace automated testing

D. To delete unused registry images

Correct Answer: A. To record exact dependency versions resolved for a build

Question 4

Why should production container deployments commonly use an image digest?

A. A digest makes the image execute faster.

B. A digest identifies content more precisely than a mutable tag.

C. A digest removes the need for a registry.

D. A digest guarantees that the image has no vulnerabilities.

Correct Answer: B. A digest identifies content more precisely than a mutable tag.

Question 5

Which item is most directly associated with software provenance?

A. The source commit and build pipeline that produced an artifact

B. The color of a deployment dashboard

C. The number of developers in a team

D. The physical size of a laptop

Correct Answer: A. The source commit and build pipeline that produced an artifact

Question 6

What is a transitive dependency?

A. A dependency explicitly installed by an operating-system administrator

B. A dependency introduced through another dependency

C. A dependency that is never used at runtime

D. A dependency stored without a version

Correct Answer: B. A dependency introduced through another dependency

Question 7

Which control helps reduce dependency-confusion risk?

A. Allowing every build to use any public registry

B. Using private namespaces and approved package sources

C. Disabling all dependency updates permanently

D. Using the tag latest for every package

Correct Answer: B. Using private namespaces and approved package sources

Question 8

What does the practice of build once and promote many times seek to ensure?

A. Each environment receives a separately rebuilt artifact

B. The tested artifact is the same artifact deployed later

C. Only development artifacts are retained

D. Dependencies are never scanned

Correct Answer: B. The tested artifact is the same artifact deployed later

Question 9

Which statement about an SBOM is correct?

A. It is an inventory of software components in a product.

B. It guarantees that every component is secure.

C. It replaces the need for access control.

D. It is only useful for user-interface applications.

Correct Answer: A. It is an inventory of software components in a product.

Question 10

Which registry practice best supports least privilege?

A. Giving all developers permission to delete production releases

B. Allowing production systems to publish arbitrary packages

C. Separating developer publishing, release promotion, and administrative permissions

D. Sharing one permanent administrator password

Correct Answer: C. Separating developer publishing, release promotion, and administrative permissions