CI/CD and DevOps Foundations

Introduction

Modern software organisations compete not only through the features they develop, but also through how reliably and rapidly they deliver those features. Traditional delivery models often separated development, testing, operations, security, and business teams. Each group optimised its own activities, frequently producing delays, hand-off errors, unclear accountability, and difficult releases. DevOps emerged as a socio-technical response to these problems.

DevOps is not merely a toolset and is not synonymous with Jenkins, GitHub Actions, containers, or cloud computing. It is a way of organising people, processes, technology, and feedback so that software can move safely from idea to operation. Continuous integration and continuous delivery or deployment are central engineering practices within this approach. They provide mechanisms for integrating work frequently, validating changes automatically, and releasing software through a repeatable process.

This chapter establishes the foundations required for later practical work with Jenkins and GitHub Actions. It explains DevOps culture, value streams, continuous integration, continuous delivery, continuous deployment, feedback loops, the CALMS framework, and the relationship between DevOps, Agile, and site reliability engineering (SRE).

Learning Objectives

By the end of this chapter, you should be able to:

1. Explain DevOps as a cultural and technical operating model.
2. Distinguish continuous integration, continuous delivery, and continuous deployment.
3. Describe how value-stream thinking and feedback loops improve delivery.
4. Apply the CALMS framework to a software-delivery environment.
5. Explain the relationship between Agile, CI/CD, DevOps, and SRE.
6. Identify common implementation risks and propose suitable controls.

1. From Siloed Delivery to DevOps

A software product passes through a value stream: a sequence of activities that transforms a customer need into a usable service or product. Typical activities include discovery, design, coding, review, testing, packaging, deployment, operation, monitoring, and learning. In a siloed organisation, each activity may be owned by a separate department. Work waits in queues between departments, and information about production behaviour may not return to developers.

DevOps seeks to reduce these organisational and technical barriers. It encourages shared responsibility for the complete service lifecycle. Developers consider operability and security; operations teams participate in design and automation; testers contribute early; and product teams use operational evidence when prioritising work.

DevOps therefore has three connected dimensions:

• Culture: trust, collaboration, shared ownership, learning, and psychological safety.
• Practices: version control, automated testing, continuous integration, infrastructure as code, progressive delivery, monitoring, and incident learning.
• Technology: platforms and tools that make these practices repeatable, observable, and scalable.

Technology can support DevOps, but it cannot compensate for blame, unclear ownership, or unrealistic delivery policies. Installing a pipeline tool without changing the underlying workflow may simply automate an inefficient process.

Important properties of DevOps culture

Shared goals replace local optimisation. Teams measure customer and service outcomes rather than only the number of tickets completed. Small batches reduce the amount of change exposed to risk. Automation removes repetitive manual work and improves consistency. Transparency makes work status, test results, deployment state, and incidents visible. Continuous learning treats failures and near misses as opportunities to improve systems rather than occasions for personal blame.

A blameless culture does not mean the absence of accountability. It means that investigations focus first on system conditions, decisions, communication, and controls. Individuals remain responsible for professional conduct, but the organisation also accepts responsibility for designing safe processes.

2. Value Streams and Flow

Value-stream thinking asks how an idea becomes customer value and where that journey is delayed or degraded. A value-stream map may show queues, hand-offs, approval steps, rework, testing constraints, deployment activities, and operational feedback.

Four common flow problems are:

1. Waiting: work is complete but remains in a queue for review, testing, or deployment.
2. Handoffs: important knowledge is lost as work moves between specialised groups.
3. Batching: large releases accumulate changes, increasing integration and failure risk.
4. Rework: defects discovered late require repeated design, coding, and testing effort.

CI/CD practices improve flow by reducing batch size, automating verification, and making release steps repeatable. However, a fast pipeline is not automatically valuable. It must deliver secure, reliable software that satisfies a genuine user or business need.

A useful operational perspective is to monitor both flow and outcomes. Common measures include deployment frequency, lead time for changes, change failure rate, and time to restore service. These measures are often associated with the DORA research programme. They should be used to understand system performance and guide improvement, not to rank individuals or encourage unsafe competition.

3. Continuous Integration

Continuous integration (CI) is the practice of integrating code changes into a shared repository frequently, followed by automated compilation, testing, and other checks. The purpose is to discover integration problems close to the time they are introduced.

A typical CI process is:

1. A developer creates a small change in a version-controlled branch.
2. The change is submitted through a pull request or equivalent mechanism.
3. The CI server checks out the source code.
4. Dependencies are resolved using controlled versions.
5. The software is built or packaged.
6. Unit, integration, static-analysis, security, and policy checks run.
7. Results are reported to the team.
8. A successful revision may produce a versioned artifact.

CI is a behaviour and a discipline, not simply a server job. If developers integrate only once near the end of a project, a tool labelled “CI” does not create continuous integration.

Characteristics of effective CI

Effective CI generally includes a single authoritative source repository, automated and repeatable builds, tests that provide meaningful feedback, short execution time for essential checks, and visible results. Build environments should be reproducible. Dependencies should be pinned or governed so that a build does not silently change because an external package changed.

A healthy CI system separates fast feedback from slower validation. For example, unit tests may run on every commit, while extensive end-to-end tests run after a merge or before release. The separation must not become an excuse to ignore slow or unreliable tests. Flaky tests should be investigated because they reduce trust in the pipeline.

CI is more than testing

Testing is essential, but CI may also validate formatting, licensing, dependency vulnerabilities, infrastructure definitions, database migrations, accessibility, and compliance policies. The appropriate checks depend on the product and its risks. A medical or financial system requires stronger controls than a low-risk internal prototype, although both benefit from early automated feedback.

4. Continuous Delivery and Continuous Deployment

The terms continuous delivery and continuous deployment are related but distinct.

Continuous delivery means that every change that passes the required automated and human controls is kept in a releasable condition. The organisation may choose when to release it, perhaps because of market timing, regulatory review, customer coordination, or operational risk. A manual approval can exist in continuous delivery, provided the software is technically ready for release.

Continuous deployment goes further: every change that passes the defined pipeline controls is automatically deployed to production, without a routine manual release decision.

The distinction can be represented as follows:

Continuous deployment is not inherently more mature than continuous delivery. It may be unsuitable where legal approval, scheduled releases, physical installation, or exceptional operational risk is involved. The important principle is that the delivery process should be reliable, observable, and proportionate to risk.

Deployment strategies

A pipeline may deploy using several strategies:

• Rolling deployment: instances are replaced gradually.
• Blue-green deployment: traffic moves between two production environments.
• Canary deployment: a small proportion of users receive the change first.
• Feature flags: code is deployed while functionality is enabled selectively.
• Recreate deployment: the old version is stopped before the new version starts; this is simple but may cause downtime.

These strategies reduce blast radius, but they do not eliminate the need for testing, monitoring, rollback planning, or data-compatibility analysis.

5. Feedback Loops and the Delivery System

A feedback loop is a path through which information about an action returns to influence a later decision. CI provides rapid feedback about code quality. Deployment monitoring provides feedback about runtime behaviour. Customer research provides feedback about whether the product solves a meaningful problem. Incident reviews provide feedback about resilience and organisational weaknesses.

Feedback is most useful when it is:

• Fast: delivered while the responsible team can still act.
• Relevant: connected to a decision or risk.
• Reliable: based on trustworthy data.
• Visible: accessible to those who need it.
• Actionable: capable of producing a specific improvement.

A pipeline that reports only “failed” is less useful than one that identifies the failed test, affected revision, logs, environment, and suggested owner. Similarly, monitoring only CPU usage may fail to reveal that customers cannot complete payments. Technical signals should be connected to user and business outcomes.

The DevOps feedback cycle can be understood as a loop: plan, develop, integrate, verify, release, operate, observe, and learn. The loop should not be interpreted as a rigid sequence; work often moves through parallel activities. Its central idea is that delivery and operation generate knowledge, and that knowledge should influence the next change.

6. The CALMS Framework

CALMS is a widely used framework for discussing DevOps capability. The letters represent Culture, Automation, Lean, Measurement, and Sharing. It is a conceptual framework rather than a compliance checklist.

Culture

Culture determines whether teams use automation and measurement to learn or to assign blame. Leaders influence culture through incentives, workload expectations, resource allocation, and their response to failure.

Automation

Automation should be version controlled, tested, observable, and recoverable. An automated process that cannot be understood or safely changed may create hidden operational risk. Secrets must not be placed in source code or unprotected pipeline logs.

Lean

Lean thinking focuses on value, flow, pull, and the elimination of waste. Examples of waste include waiting for an unnecessary approval, building features that users do not need, and repeating manual environment setup.

Measurement

Measurements should support learning. Useful measures include delivery flow, reliability, quality, security, cost, and user experience. A single metric can be gamed or misunderstood; a balanced set is more informative.

Sharing

Sharing includes documentation, communities of practice, reusable pipeline components, post-incident learning, and transparent dashboards. It also includes sharing responsibility: teams should not conceal failures because another group technically owns the affected system.

7. Relationship Between Agile, DevOps, and SRE

Agile is an approach to developing products under conditions of changing requirements and uncertainty. Agile methods encourage incremental delivery, customer collaboration, working software, and adaptation. Scrum, Kanban, and Extreme Programming are examples of approaches associated with Agile, although they emphasise different practices.

DevOps extends the flow beyond development. Agile may help a team deliver increments of functionality, while DevOps connects those increments to testing, release, operation, monitoring, and organisational learning. Agile without operational feedback can optimise coding while neglecting reliability. DevOps without product discovery can deliver unwanted software efficiently.

Site reliability engineering (SRE) is an engineering discipline for operating reliable services. It treats operations as a software and systems problem and uses automation, monitoring, capacity planning, incident response, and reliability objectives. SRE introduces concepts such as:

• Service-level indicator (SLI): a measured aspect of service performance, such as successful request proportion.
• Service-level objective (SLO): a target level for an SLI over a specified period.
• Service-level agreement (SLA): an external commitment that may include consequences if targets are missed.
• Error budget: the tolerated amount of unreliability implied by an SLO.

For example, if an SLO requires 99.9% successful requests over a period, the permitted unsuccessful proportion is 0.1%. This does not mean that every system must use this target; the target must reflect user needs, technical feasibility, and business consequences. Error budgets help balance feature delivery and reliability work. If the budget is being consumed too quickly, the organisation may pause risky releases and invest in reliability.

Agile, DevOps, and SRE overlap but are not interchangeable:

8. CI/CD Pipeline Architecture and Governance

A CI/CD pipeline is a controlled sequence of automated and sometimes manual stages. A practical pipeline may contain source checkout, dependency installation, linting, unit tests, build, security scanning, packaging, integration tests, deployment to a test environment, acceptance checks, approval or policy evaluation, production deployment, and post-deployment verification.

Jenkins and GitHub Actions are examples of platforms that can implement such workflows. Jenkins is commonly operated as a configurable automation server with agents and plugins. GitHub Actions represents workflows as configuration associated with repositories and uses hosted or self-hosted runners. The tools differ in administration, integration, security model, and cost, but the foundational principles remain the same: version-controlled pipeline definitions, least privilege, reproducibility, visible results, and controlled promotion of artifacts.

Governance controls

Professional pipelines should address:

• authentication and role-based access;
• protection of production branches and environments;
• secret storage and rotation;
• dependency and artifact provenance;
• audit logs and approval records;
• separation of duties where required;
• vulnerability handling and patching;
• rollback or forward-fix procedures;
• retention and privacy requirements.

Security should be integrated into the pipeline rather than postponed until release. This approach is often called DevSecOps. Nevertheless, automation does not make a decision automatically correct. Security findings require triage, and risk acceptance should be explicit and authorised.

9. Illustrative Case: Northstar Retail Services

Background. Northstar Retail Services operates an online ordering platform. Developers release approximately once every six weeks. Operations receives release packages manually, and production incidents are difficult to diagnose because test and production configurations differ.

Problem. Releases contain many unrelated changes. Testing begins late, deployment depends on a small number of specialists, and developers receive little information about customer impact. A failed release can take several hours to reverse.

Analysis. A value-stream review identifies long approval queues, manual environment configuration, large batches, and missing runtime dashboards. The organisation also discovers that its measure of success—number of features completed—does not capture reliability or delivery delay.

Approach. Northstar first places application code and pipeline definitions under version control. It creates an automated CI workflow with unit tests, static analysis, dependency checks, and reproducible packaging. The team then creates an environment that resembles production and adds integration tests. Deployments use a canary strategy, with automated checks for error rate and checkout completion. A failed canary stops promotion. Secrets are moved to a managed secret store, and production access is limited through roles. Developers and operations staff conduct a blameless review after each incident.

The company chooses continuous delivery rather than immediate continuous deployment because payment-related changes require a documented business approval. However, the approval occurs after the software has passed all technical checks and is ready to release.

Outcome. Release batches become smaller, the time required to prepare a release falls, and failures are detected earlier. The organisation does not eliminate every incident, but it reduces the impact and improves recovery. Teams also learn that pipeline speed alone is insufficient: reliable tests, useful monitoring, and safe operational practices are equally important.

Learning points. Northstar improves by changing the system of work rather than purchasing a tool. It uses incremental adoption, measures outcomes, treats security as part of delivery, and selects a release model appropriate to business risk.

10. Challenges and Limitations

DevOps adoption can fail when it is treated as a rebranding exercise, a demand for unlimited automation, or a justification for reducing staff without reducing risk. Common challenges include legacy architecture, inadequate test coverage, unstable environments, limited skills, resistance caused by previous blame cultures, and excessive pipeline complexity.

Automation can amplify errors. A faulty deployment script may affect every environment rapidly. Continuous deployment can also increase risk if monitoring is weak or if database changes are irreversible. Microservices do not automatically produce DevOps benefits; they may increase operational complexity.

Metrics require careful interpretation. High deployment frequency may result from trivial changes, while low frequency may be appropriate for a regulated system. Lead time should be considered with quality, reliability, security, and customer outcomes. Teams should avoid setting targets that encourage bypassing tests or concealing failures.

Artificial intelligence can assist with log summarisation, test generation, anomaly detection, and workflow recommendations. However, AI-generated code and pipeline changes require review, testing, licence consideration, privacy protection, and security assessment. AI systems may produce incorrect explanations or expose sensitive information through prompts and outputs. Human accountability remains necessary for production decisions.

Best-Practice Adoption Path

A diploma-level practitioner can approach CI/CD adoption incrementally:

1. Map the value stream and identify the most costly delay or failure.
2. Place source code, configuration, and pipeline definitions under version control.
3. Establish a reproducible build.
4. Add fast, trustworthy automated tests.
5. Produce versioned artifacts and promote the same artifact between environments.
6. Add security, quality, and compliance checks proportionate to risk.
7. Automate deployment to a non-production environment.
8. Introduce monitoring, rollback, and post-deployment verification.
9. Measure flow and service outcomes.
10. Improve collaboratively through regular retrospectives and incident learning.

The goal is not automation for its own sake. The goal is a delivery system that enables frequent, safe, observable, and valuable change.

Chapter Summary

DevOps is a socio-technical approach that joins collaboration, automation, measurement, and learning across the software lifecycle. Value-stream thinking reveals queues, hand-offs, batching, and rework that obstruct delivery. Continuous integration integrates changes frequently and validates them automatically. Continuous delivery keeps validated changes ready for release, whereas continuous deployment releases each qualifying change automatically.

Feedback loops connect development, delivery, operation, customers, and organisational learning. CALMS—Culture, Automation, Lean, Measurement, and Sharing—provides a useful framework for assessing DevOps capability. Agile supports adaptive product development; DevOps extends attention across the delivery and operational lifecycle; and SRE applies engineering practices to service reliability through indicators, objectives, error budgets, automation, and incident learning.

Effective CI/CD requires more than a tool. It requires reproducible processes, secure governance, meaningful tests, observability, appropriate release strategies, and a culture that treats failure as information. Jenkins and GitHub Actions can implement these practices, but the underlying principles apply regardless of platform.

Key Terms

Agile: An adaptive approach to product development based on incremental delivery, collaboration, and responding to change.

Artifact: A versioned output of a build, such as a package, container image, or compiled application.

CALMS: A DevOps framework representing Culture, Automation, Lean, Measurement, and Sharing.

Canary deployment: Releasing a change initially to a small portion of users or infrastructure.

Continuous delivery: Keeping every validated change in a releasable state.

Continuous deployment: Automatically releasing every change that passes required controls to production.

Continuous integration: Frequently integrating code and automatically validating the integrated result.

DevOps: A socio-technical approach to improving software delivery and operation through collaboration, automation, feedback, and shared responsibility.

Error budget: The tolerated unreliability implied by a service-level objective.

Feedback loop: A flow of information from an activity or outcome back into future decisions.

Infrastructure as code: Managing infrastructure through version-controlled, machine-readable definitions.

Lead time: The elapsed time between the start of work and delivery of the resulting change.

Pipeline: An automated or partly automated sequence of activities that builds, verifies, packages, and delivers software.

Progressive delivery: Gradually exposing a change using techniques such as canaries, feature flags, or staged rollout.

Service-level agreement: An external service commitment, often including contractual consequences.

Service-level indicator: A quantitative measure of a service attribute, such as availability or latency.

Service-level objective: A target value or range for a service-level indicator over a defined period.

Site reliability engineering: An engineering discipline focused on reliable, scalable, and sustainable operation of services.

Value stream: The activities and information flows that transform a need into a delivered outcome.

Review Questions

Short Answer Questions

1. What is the primary purpose of continuous integration?
2. How does continuous delivery differ from continuous deployment?
3. Name the five elements of the CALMS framework.
4. What is a feedback loop in a DevOps system?
5. Define a service-level objective.

Long Answer Questions

1. Explain how DevOps culture changes responsibilities between development, testing, security, and operations teams.
2. Describe a complete CI/CD pipeline and explain the purpose of each major stage.
3. Compare Agile, DevOps, and SRE, including the ways in which they complement one another.
4. Explain how value-stream mapping can reveal delivery waste and suggest suitable improvements.
5. Discuss the governance and security controls required for a professional CI/CD pipeline.

Analytical / Application Questions

1. A team deploys frequently but experiences frequent production failures and has no reliable monitoring. Analyse the weaknesses in its DevOps system and propose a staged improvement plan.
2. A bank requires documented approval for payment-service releases but wants faster delivery. Decide whether continuous delivery or continuous deployment is more appropriate and justify your decision.
3. A team wants to measure performance using deployment frequency alone. Evaluate this proposal and recommend a balanced set of delivery and reliability measures.

Multiple Choice Questions with Answers

Question 1

What is the central purpose of continuous integration?

A. To integrate and validate changes frequently

B. To eliminate all production monitoring

C. To replace version control with manual packaging

D. To require every change to be deployed directly to production

Correct Answer: A. To integrate and validate changes frequently

Question 2

Which statement best describes continuous delivery?

A. Every change is automatically released to all users

B. Validated changes are maintained in a releasable state

C. Developers integrate code only at the end of a project

D. Operations performs all testing manually

Correct Answer: B. Validated changes are maintained in a releasable state

Question 3

In the CALMS framework, what does the letter L represent?

A. Logging

B. Licensing

C. Lean

D. Leadership

Correct Answer: C. Lean

Question 4

Which practice most directly reduces the risk of a large release batch?

A. Integrating small changes frequently

B. Delaying all testing until production

C. Combining unrelated features into one release

D. Removing version control

Correct Answer: A. Integrating small changes frequently

Question 5

What is the main purpose of a canary deployment?

A. To expose a change gradually to a limited audience or infrastructure

B. To prevent all automated testing

C. To replace monitoring with manual inspection

D. To ensure that two unrelated applications share one database

Correct Answer: A. To expose a change gradually to a limited audience or infrastructure

Question 6

Which item is an example of an SLI?

A. A target of 99.9% successful requests

B. The measured proportion of successful requests

C. A contractual penalty for missing a target

D. A team’s annual training budget

Correct Answer: B. The measured proportion of successful requests

Question 7

How does SRE primarily contribute to DevOps?

A. By focusing engineering practices on reliable service operation

B. By removing the need for incident response

C. By requiring that every organisation use one programming language

D. By replacing customer feedback with infrastructure metrics

Correct Answer: A. By focusing engineering practices on reliable service operation

Question 8

Which pipeline practice improves artifact consistency between environments?

A. Rebuilding the application separately in every environment

B. Promoting the same versioned artifact between environments

C. Editing binaries manually before production

D. Downloading unpinned dependencies during deployment

Correct Answer: B. Promoting the same versioned artifact between environments

Question 9

Why should CI/CD metrics not be used alone to rank individuals?

A. Metrics are always mathematically invalid

B. Metrics can be gamed and may ignore quality, reliability, and customer outcomes

C. Delivery systems cannot produce any measurements

D. Individuals never influence software delivery

Correct Answer: B. Metrics can be gamed and may ignore quality, reliability, and customer outcomes

Question 10

A team has automated builds but integrates code only once every six weeks. Which conclusion is most accurate?

A. It has fully achieved continuous integration

B. It has automation, but not the frequent integration behaviour required for CI

C. It has continuous deployment

D. It has implemented SRE automatically

Correct Answer: B. It has automation, but not the frequent integration behaviour required for CI