Build Automation, Dependencies, and Artifacts

Introduction

A CI/CD pipeline is only as reliable as the build it produces. Source code may be correct, tests may pass, and deployment automation may be well designed; nevertheless, a release can fail if dependencies change unexpectedly, generated files are not preserved, or the published package cannot be traced to the source that created it. Build automation therefore concerns more than executing a compiler. It establishes a repeatable process for transforming source code and declared inputs into verified, identifiable, and distributable outputs.

This chapter examines build automation in Jenkins and GitHub Actions, with emphasis on reproducibility, dependency management, caching, package creation, artifact versioning, provenance, retention, and publication to package and container registries. The discussion applies to common ecosystems such as Java/Maven, JavaScript/npm, Python, and containerized applications.

Learning Objectives

By the end of this chapter, students should be able to:

• explain the relationship among source, dependencies, build tools, artifacts, and registries;
• design a reproducible build using pinned tools and locked dependencies;
• distinguish dependency caching from build-artifact storage;
• create, name, version, test, and publish packages and container images;
• apply retention, access-control, integrity, and provenance practices;
• implement a practical Jenkins or GitHub Actions build pipeline; and
• evaluate the limitations and governance risks of automated artifact production.

1. The Build as a Controlled Transformation

A useful abstraction is:

The function is conceptual rather than a claim that every build is mathematically deterministic. Source includes application code and build scripts; Dependencies includes direct and transitive libraries; Toolchain includes compilers, interpreters, package managers, and plugins; Configuration includes build flags and environment-specific settings; and Environment includes the operating system, architecture, and relevant system libraries.

A build is reproducible when the same declared inputs produce equivalent output under the documented build procedure. It is hermetic when the procedure cannot silently obtain undeclared inputs from the network, host machine, or mutable external state. Reproducibility is a spectrum: byte-for-byte identity is stronger than functional equivalence, and functional equivalence is stronger than merely obtaining a successful build.

A pipeline should make the following dependencies explicit:

A build that uses latest container images, floating dependency ranges, or an unrecorded operating-system package is not fully controlled, even if it usually succeeds.

2. Build Automation in Jenkins and GitHub Actions

Jenkins commonly represents a pipeline in a Jenkinsfile. Agents provide execution environments, while stages express activities such as checkout, dependency restoration, compilation, testing, packaging, scanning, and publication. Credentials should be supplied through Jenkins Credentials Binding or an approved secret-management integration, never hard-coded in the repository.

GitHub Actions represents a workflow in YAML under .github/workflows. A workflow is triggered by events such as a push, pull request, tag, or manual dispatch. Jobs run on hosted or self-hosted runners and may exchange files through uploaded artifacts. Actions should be pinned to trusted commit SHAs where supply-chain risk is significant, rather than relying only on mutable tags.

A typical build flow is:

Figure: A controlled build flow from immutable source to a published artifact.

The stages should be separated logically even when they run in one job. This separation improves diagnosis, permits approvals before publication, and makes it possible to enforce policies such as “only tagged commits may publish production artifacts.”

3. Dependency Management

A direct dependency is explicitly requested by a project. A transitive dependency is required by another dependency. Transitive libraries can introduce vulnerabilities, license obligations, or incompatible versions; therefore, dependency management is a supply-chain responsibility rather than a convenience feature.

Good practice includes:

1. Declare dependencies in a manifest such as pom.xml, package.json, pyproject.toml, or a Go module file.
2. Commit the lock or resolution file, such as package-lock.json, poetry.lock, or requirements.txt with hashes where supported.
3. Prefer exact versions for production builds. Ranges may be useful for libraries, but they reduce determinism in applications.
4. Verify checksums or signatures when the ecosystem supports them.
5. Run vulnerability and license checks as quality gates.
6. Update dependencies through reviewed pull requests and record the reason for exceptions.

A lockfile records a resolved dependency graph. It does not guarantee safety: a locked vulnerable package remains vulnerable. Nor does it guarantee cross-platform identity if optional dependencies or native compilation differ by operating system and architecture.

Dependency resolution should occur before compilation and should fail when the lockfile is inconsistent with the manifest. For example, a CI command that silently rewrites a lockfile can conceal an unintended change. A safer process is to use a frozen or locked install mode in CI.

Dependency Confusion and Trust

Registries may contain packages with the same name in different scopes. An attacker can publish a public package whose name resembles an internal package and rely on resolver precedence. Organizations should use explicit package scopes, private registries, approved dependency sources, and repository-level configuration that prevents accidental fallback to public sources. Credentials for private registries must be limited to read access during ordinary builds.

4. Caching: Speed Without Losing Correctness

Caching avoids repeatedly downloading or rebuilding unchanged inputs. A dependency cache may contain Maven modules, npm packages, Python wheels, or compiler toolchains. A build cache may contain intermediate compilation results. A pipeline artifact is different: it is a deliberate output retained for later jobs, review, deployment, or audit.

The central rule is that a cache is disposable and potentially stale; an artifact is a named product with a defined retention and integrity expectation. A cache miss should slow a build, not change its result. If deleting a cache changes correctness, the cache is incorrectly being used as storage for required inputs.

A cache key should represent every input that can affect cached content. A common conceptual form is:

Here, ∥ means concatenation with an unambiguous separator. The key is applicable only when those variables adequately capture the cache’s dependencies; additional compiler flags, Python versions, or package-manager settings may also be required. A restore key or prefix can permit a partial match, but the build must still validate the result and must not treat a stale partial cache as authoritative.

In GitHub Actions, actions/cache is appropriate for package-manager caches, while upload-artifact stores job outputs. In Jenkins, caches may be implemented through persistent workspaces, mounted volumes, or a remote cache system. Persistent workspaces require careful cleanup because files from a previous build can contaminate a later one. A clean checkout and isolated workspace are safer defaults.

5. Package Creation and Artifact Identity

An artifact is a deployable or reusable output: a JAR, npm package, Python wheel, executable, archive, Helm chart, or container image. An artifact should have:

• a stable name;
• a version;
• metadata identifying its source and build;
• a checksum or content digest;
• a documented format and target platform; and
• a retention and publication policy.

Versioning

Semantic Versioning commonly expresses a version as MAJOR.MINOR.PATCH. Under its usual interpretation, incompatible public API changes increase MAJOR, backward-compatible functionality increases MINOR, and backward-compatible fixes increase PATCH. Pre-release identifiers, such as 2.0.0-rc.1, indicate that the version is not necessarily final. Semantic Versioning is a communication convention, not a guarantee that an API is actually compatible.

A practical pipeline may derive a release version from a signed or protected Git tag. Pull-request builds can use a non-publishable identifier, while release builds use the tag. Avoid rebuilding a release with the same version after publication. If the contents change, the version should change; otherwise consumers cannot reliably distinguish the outputs.

A filename is not a sufficient identity. For a container image, the immutable identity is its digest, commonly represented as sha256:...; a tag such as 1.4.0 is a human-friendly reference that can be moved unless registry policy prevents it. Deployment records should therefore retain both tag and digest.

Integrity

For a file, a cryptographic digest is commonly calculated as:

h is a 256-bit digest of the exact byte sequence fileBytes. The digest detects accidental or unauthorized modification when compared through a trusted channel, but it does not by itself prove who produced the file. Authenticity requires a trusted signature or provenance system, and secrecy requires encryption.

6. Provenance and Supply-Chain Security

Provenance describes where an artifact came from and how it was produced. Useful fields include source repository and commit, workflow or Jenkins job, builder identity, build timestamp, toolchain, dependency information, parameters, and artifact digest. Modern systems may express this using an attestation format such as an in-toto statement or a SLSA-oriented provenance record.

Provenance supports several questions:

• Which source commit produced this package?
• Was it built by an approved workflow?
• Which dependencies and builder image were used?
• Has the artifact been rebuilt or replaced?
• Can a vulnerable library be traced to deployed services?

A signed provenance statement improves confidence, but it does not make insecure source code safe. Trust depends on the identity of the signer, protection of the signing key, correctness of the build system, and review of the declared inputs. Short-lived identity federation is generally preferable to long-lived registry passwords or static signing keys.

Build logs can contain secrets accidentally printed by commands or tools. Masking reduces exposure but is not perfect. Pipelines should use least-privilege tokens, restrict publication permissions, prevent untrusted pull requests from accessing secrets, and review third-party actions and Jenkins plugins.

7. Publication to Registries

A package registry stores language-specific packages and metadata. A container registry stores image manifests and layers. Publication should normally occur only after compilation, tests, security checks, and policy validation succeed.

A robust publication sequence is:

1. Authenticate using a short-lived, least-privilege credential.
2. Build once in a controlled job.
3. Test and scan the exact output that will be published.
4. Attach metadata and provenance.
5. Publish using a unique version or immutable reference.
6. Record the registry URL and digest.
7. Verify that the registry contains the expected artifact.
8. Promote the same artifact between environments rather than rebuilding it.

The last principle is important. Rebuilding separately for staging and production can produce different outputs if dependencies, timestamps, tool versions, or network responses change. Promotion preserves the tested artifact.

Illustrative GitHub Actions Workflow

The following simplified example demonstrates the order of operations. It is illustrative; action versions, permissions, registry names, and secret names must be adapted to the organization.

name: build
on:
  push:
    branches: [main]
    tags: ['v*']
permissions:
  contents: read
  packages: write
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: npm
      - run: npm ci
      - run: npm test
      - run: npm pack
      - uses: actions/upload-artifact@v4
        with:
          name: package-${{ github.sha }}
          path: '*.tgz'
      - if: startsWith(github.ref, 'refs/tags/v')
        run: npm publish
        env:
          NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}

In production, the workflow should additionally verify the tag, scan dependencies, restrict publishing to protected tags, avoid exposing tokens to untrusted code, and pin critical actions by commit SHA where organizational policy requires it.

Jenkins Pattern

A Jenkins declarative pipeline typically uses stages such as Checkout, Resolve Dependencies, Test, Package, and Publish. The publish stage should be conditional on a release tag or approved branch and should use credentials injected only for that stage. Workspace cleanup should occur before or after execution, and the agent image should be versioned rather than silently changing.

8. Retention and Lifecycle Management

Retention is both a cost and governance issue. Keeping every nightly build forever increases storage and may preserve vulnerable or sensitive material. Deleting too aggressively can undermine incident investigation, rollback, or regulatory obligations.

A policy should distinguish:

Retention must not be confused with deletion of a released version that clients still depend on. Package immutability and deprecation policies should be defined before publishing. Vulnerable artifacts may need quarantine or blocking, but emergency deletion can break downstream systems; coordinated withdrawal is preferable.

9. Detailed Applied Case: Northstar Payments

Background

Northstar Payments operates a Java service and a container image. Its Jenkins pipeline builds on shared agents and publishes to an internal Maven registry and container registry.

Problem

Teams reported that a release passed staging but behaved differently in production. Investigation found three causes: the build used a floating base image tag, Maven dependencies were cached in a shared workspace, and the same version number was republished after a source change. The organization could not determine which commit produced the production image.

Analysis

The pipeline had no explicit toolchain version, no lock or verification policy for all inputs, and no immutable release identity. A cache hit could contain files from another branch. The image tag was recorded, but the registry had allowed that tag to move. Tests were run before packaging, but the exact published image was not scanned or retained as a pipeline artifact.

Approach and Solution

Northstar introduced the following controls:

• a versioned builder image containing the approved JDK and Maven versions;
• clean, isolated workspaces for every build;
• dependency verification and a reviewed dependency-update process;
• cache keys incorporating operating system, architecture, toolchain, and dependency-lock content;
• release versions generated from protected Git tags;
• a rule preventing reuse of a published version;
• image publication by immutable digest, with the digest recorded in the deployment system;
• vulnerability scanning of the final image rather than only the source repository;
• signed provenance containing commit, builder, dependencies, and workflow identity; and
• separate retention schedules for pull-request outputs, snapshots, releases, and audit evidence.

Outcome

The team could reproduce a release from its commit and identify the exact image deployed. Build time decreased because dependency caching was made safe rather than shared indiscriminately. Rollback became a promotion of a known digest, not a new build. The organization also identified that provenance records did not replace code review or vulnerability remediation; they provided evidence, not automatic security.

Learning Points

The case demonstrates that reliability comes from controlling the entire input boundary. Caching, versioning, and provenance are connected: a fast build with ambiguous inputs is not a dependable build, and a uniquely named artifact without evidence of origin is not fully auditable.

10. Challenges, Limitations, and Governance

Reproducibility can be difficult when native libraries, timestamps, locale settings, file ordering, random identifiers, or external services influence output. Some build tools embed timestamps, making byte-for-byte comparison difficult. Where deterministic output is impractical, teams should document the expected equivalence, normalize timestamps where supported, and record complete provenance.

No cache design can solve an incorrectly declared dependency graph. No checksum proves that the original package was trustworthy. No vulnerability scanner detects every defect, and scanners can produce false positives or miss vulnerabilities in unreachable code. Human review, testing, secure design, and incident response remain necessary.

Artifact systems also raise governance questions. Proprietary code and credentials may accidentally enter logs or archives. Retention must respect privacy, contractual obligations, export restrictions, and applicable data-protection law. Access should follow least privilege, and audit records should be protected from alteration. AI-assisted code generation introduces additional concerns: generated code must undergo the same review, license analysis, testing, and provenance procedures as human-written code; prompts or source material must not disclose confidential information.

11. Contemporary Practice and Best Practices

Current professional practice increasingly combines package registries, container registries, software bills of materials (SBOMs), signed attestations, dependency automation, and policy-as-code. An SBOM lists components and versions; it does not itself prove that the list is complete or that components are safe. Automated update tools can create pull requests, but maintainers must review compatibility and test results.

A practical checklist is:

• Pin runner, builder, action, and dependency versions where appropriate.
• Commit and enforce lockfiles.
• Build in clean, isolated environments.
• Treat caches as disposable accelerators.
• Test and scan the exact artifact to be published.
• Use unique versions and immutable digests.
• Publish only from trusted, protected release paths.
• Generate provenance and, where feasible, an SBOM.
• Use short-lived credentials and least privilege.
• Retain releases, logs, and evidence according to risk and policy.
• Promote tested artifacts instead of rebuilding for each environment.
• Monitor registry access and review dependencies continuously.

Chapter Summary

Build automation transforms source, dependencies, tools, configuration, and environment into a distributable artifact. Reliable pipelines make these inputs explicit, use locked and verified dependencies, and execute in controlled environments. Caches improve speed but must remain disposable and must never replace artifact storage. Packages and images require stable names, meaningful versions, checksums or digests, and appropriate retention. Provenance connects an artifact to its source and builder, while signatures and attestations strengthen—not replace—trust controls. Jenkins and GitHub Actions can implement these practices through staged workflows, protected publication paths, isolated runners, and carefully managed credentials. The Northstar case illustrates why reproducibility, immutability, and evidence must be designed together.

Key Terms

Artifact: A build output intended for distribution, deployment, or reuse.

Build reproducibility: The ability to obtain equivalent output from the same declared inputs and procedure.

Hermetic build: A build restricted to explicitly declared inputs.

Dependency: A library, tool, or component required by a project.

Direct dependency: A dependency explicitly declared by the project.

Transitive dependency: A dependency introduced through another dependency.

Lockfile: A file recording resolved dependency versions and related metadata.

Cache: Temporary stored data used to accelerate repeated work.

Package registry: A service that stores and distributes language-specific packages.

Container registry: A service that stores container image manifests and layers.

Semantic Versioning: A convention using major, minor, and patch version components.

Digest: A content-derived cryptographic identifier.

Tag: A mutable or immutable human-readable reference to an artifact.

Provenance: Evidence describing an artifact’s source, builder, inputs, and process.

Attestation: A signed or otherwise verifiable statement about an artifact.

SBOM: A software bill of materials listing software components and versions.

Retention: The period and conditions under which artifacts and evidence are stored.

Promotion: Moving the same tested artifact between environments without rebuilding.

Dependency confusion: An attack involving misleading package names or registry precedence.

Review Questions

Short Answer Questions

1. What is the difference between a reproducible build and a hermetic build?
2. Why should a lockfile be committed to version control?
3. How does a cache differ from a pipeline artifact?
4. Why is a container digest more reliable than a mutable image tag?
5. Name three fields that should appear in build provenance.

Long Answer Questions

1. Explain how a Jenkins or GitHub Actions pipeline can control source, toolchain, dependencies, configuration, and environment.
2. Discuss the benefits and risks of dependency caching, including the design of an appropriate cache key.
3. Explain how semantic versioning, checksums, provenance, and registry immutability contribute to artifact trust.
4. Design a retention and publication policy for pull-request builds, snapshots, and production releases.
5. Evaluate why promotion of a tested artifact is generally safer than rebuilding separately for each environment.

Analytical / Application Questions

1. A build succeeds only when a shared Jenkins workspace contains an old dependency directory. Diagnose the likely problem and propose a corrective design.
2. A production container was deployed using tag 1.8.0, but the tag now points to a different image. Explain how the incident should be investigated and how future deployments should identify images.
3. Design the stages, controls, credentials, and evidence for a release pipeline that publishes a package and container image only from a protected Git tag.

Multiple Choice Questions with Answers

Question 1

Which input is most directly intended to make dependency resolution repeatable in CI?

A. A lockfile

B. A deployment log

C. A mutable container tag

D. A workspace cache

Correct Answer: A. A lockfile

Question 2

What is the principal difference between a cache and an artifact?

A. A cache is always encrypted, whereas an artifact is not

B. A cache accelerates work and is disposable, whereas an artifact is a named output retained for use or evidence

C. A cache can contain source code, whereas an artifact cannot

D. A cache is published to a registry, whereas an artifact never is

Correct Answer: B. A cache accelerates work and is disposable, whereas an artifact is a named output retained for use or evidence

Question 3

Which reference provides the strongest immutable identity for a container image?

A. The latest tag

B. A branch name

C. A content digest

D. A build directory name

Correct Answer: C. A content digest

Question 4

What should normally happen before an artifact is published?

A. The dependency cache should be deleted from the registry

B. The exact output should be tested and policy-checked

C. The release version should be reused from an earlier build

D. Credentials should be printed in the build log

Correct Answer: B. The exact output should be tested and policy-checked

Question 5

Which item is an example of transitive dependency risk?

A. A library explicitly listed by the application

B. A library required by another declared library

C. A Git commit identifier

D. A container registry URL

Correct Answer: B. A library required by another declared library

Question 6

What does a cryptographic digest primarily provide?

A. Evidence that a trusted person created the file

B. Confidentiality of the file contents

C. A way to detect changes to the exact byte sequence

D. Automatic removal of vulnerabilities

Correct Answer: C. A way to detect changes to the exact byte sequence

Question 7

Which practice best reduces dependency-confusion risk?

A. Allowing unrestricted fallback to public registries

B. Using explicit private scopes and approved dependency sources

C. Using the newest package with no review

D. Deleting the project lockfile

Correct Answer: B. Using explicit private scopes and approved dependency sources

Question 8

Why should release publication be restricted to protected tags or approved paths?

A. To prevent every developer from compiling code

B. To ensure only authorized, reviewed builds receive release identity and credentials

C. To make caches permanent

D. To eliminate the need for tests

Correct Answer: B. To ensure only authorized, reviewed builds receive release identity and credentials

Question 9

What is the main purpose of provenance?

A. To compress container layers

B. To describe where and how an artifact was produced

C. To replace dependency testing

D. To assign a random package name

Correct Answer: B. To describe where and how an artifact was produced

Question 10

Why is promoting the same artifact safer than rebuilding separately for staging and production?

A. It guarantees the artifact has no vulnerabilities

B. It avoids potential differences caused by changing inputs or environments

C. It removes the need for a registry

D. It makes version numbers unnecessary

Correct Answer: B. It avoids potential differences caused by changing inputs or environments