Integrated DevOps Capstone

Introduction and Context

An integrated DevOps capstone brings together the technical practices, governance controls, and professional evidence required to deliver software safely and repeatedly. Rather than treating source control, testing, deployment, infrastructure, security, and monitoring as separate activities, the capstone models them as one value stream.

The central idea is delivery as a governed system. A change begins with a business or operational need, is represented in version-controlled code, is tested automatically, built into an immutable artifact, deployed through controlled environments, observed in production, and evaluated using evidence. Feedback from users, monitoring, incidents, and security analysis then influences the next change.

A suitable capstone therefore asks students to demonstrate more than whether an application runs. They must show:

• how changes are reviewed and traced;
• how quality and security risks are detected;
• how infrastructure is created consistently;
• how deployment can be repeated and, where possible, reversed;
• how system behaviour is observed;
• how access, secrets, and compliance are governed; and
• how claims about performance and reliability are supported by evidence.

This chapter presents a reference approach for designing and evaluating such a solution. The tools may vary—GitHub, GitLab, Jenkins, GitHub Actions, Docker, Kubernetes, Terraform, a public cloud, Prometheus, Grafana, and OpenTelemetry are examples—but the principles are tool-independent.

Learning Objectives

After completing this chapter, a student should be able to:

1. Design an end-to-end DevOps delivery architecture.
2. Apply branching, review, tagging, and traceability practices in source control.
3. Construct a CI/CD pipeline with quality, security, and approval gates.
4. Define infrastructure as code and deploy a cloud-native application.
5. Select meaningful observability signals and service-level indicators.
6. Apply identity, secrets management, supply-chain security, and governance controls.
7. Evaluate delivery performance using evidence-based metrics.
8. Produce documentation and demonstrate a defensible capstone outcome.

1. The Capstone as a Systems Problem

A DevOps system consists of interacting socio-technical elements: people, processes, tools, infrastructure, policies, and feedback. Optimising one element can harm the whole system. For example, increasing release frequency without automated tests may increase failure rates. Adding manual approvals everywhere may reduce risk but create long queues and discourage small releases.

A useful model is the delivery chain:

Plan → Code → Build → Test → Secure → Package → Deploy → Operate → Observe → Learn

The chain is not strictly linear. Production observations can create new work; failed tests return a change to development; an incident may result in a rollback, a post-incident action, and a new control.

1.1 Desired-state and feedback models

Infrastructure and deployment systems commonly use a desired-state model. The engineer declares what should exist, such as a network, database, deployment, or replica count. An automation controller compares the declared state with actual state and performs changes to reduce the difference.

This can be represented conceptually as:

The expression is conceptual rather than a universal numerical formula. “State” may include resources, configuration, versions, and policies. The controller must also account for dependencies, safety constraints, and drift. A correct system does not blindly force changes; it plans and validates them.

1.2 The minimum viable capstone

A credible bachelor-level project should contain:

• a small but non-trivial application, preferably with an API and persistent data;
• one repository or a clearly justified multi-repository arrangement;
• automated unit, integration, and at least one acceptance or smoke test;
• a container image built from a reproducible definition;
• infrastructure represented as code;
• separate development, staging, and production configurations;
• a deployment strategy and rollback procedure;
• logs, metrics, and traces or a justified observability subset;
• security scanning and controlled secrets;
• a runbook, architecture document, and evidence portfolio.

A capstone should be deliberately bounded. A small service that is well tested, observable, and governed is academically stronger than a large system whose controls cannot be demonstrated.

2. Source Control, Work Management, and Traceability

Git provides distributed version control, but Git alone does not constitute source-control governance. The capstone should define a working agreement covering branch naming, commit expectations, pull requests, review rules, protected branches, release tags, and ownership.

A practical model is trunk-based development with short-lived branches. Developers create a branch for a small change, open a pull request, pass automated checks, obtain review, and merge into the main branch. Releases are identified by immutable tags such as v1.4.0. Long-lived branches may be justified for a maintenance line, but they increase merge and testing complexity.

2.1 Traceability chain

Each change should be traceable across:

Requirement or issue → commit → pull request → pipeline run → artifact digest → deployment → operational evidence

This chain supports accountability and diagnosis. If a production defect occurs, the team should identify the deployed artifact, the source revision that created it, the tests it passed, and the person or automation that approved deployment.

A commit message is not a substitute for a work item. Requirements should include acceptance criteria, risk classification, and, where relevant, data-protection or security implications.

2.2 Review as an engineering control

Review should examine correctness, maintainability, test adequacy, security, observability, and operational impact. Approval rules should be proportionate to risk. A documentation correction need not receive the same treatment as a change to authentication or payment processing.

Protected branches can enforce:

• successful required checks;
• at least one independent reviewer;
• no direct pushes;
• signed commits where organisational policy requires them; and
• resolution of review comments.

These controls reduce accidental changes, but they do not replace professional judgement.

3. CI/CD Pipeline Design

Continuous integration (CI) validates changes frequently. Continuous delivery keeps software in a deployable state; continuous deployment automatically releases approved changes to production. The terms must be distinguished: a pipeline can support continuous delivery without automatically deploying every change to production.

3.1 Pipeline stages

A representative pipeline is:

1. Trigger and validation: check event type, branch, permissions, and required metadata.
2. Dependency installation: use lock files and a trusted cache.
3. Static quality checks: formatting, linting, type checking, and code analysis.
4. Unit tests: test isolated behaviour quickly.
5. Integration tests: test interactions with databases, queues, or external contracts.
6. Build: create a versioned artifact or container image.
7. Security checks: scan source, dependencies, image layers, and infrastructure definitions.
8. Publish: push an immutable artifact to a controlled registry.
9. Deploy to development or staging: use environment-specific configuration.
10. Smoke and acceptance tests: confirm critical user journeys.
11. Approval or policy evaluation: apply risk and release rules.
12. Production deployment: use a controlled strategy and record evidence.
13. Post-deployment verification: inspect health, error rate, latency, and logs.

Fast feedback should occur early. Expensive tests can run after inexpensive checks, but critical security or policy failures must block the relevant stage rather than merely produce warnings.

3.2 Reproducibility and immutability

A build is reproducible when the same source revision and declared inputs produce functionally equivalent output. Practical controls include:

• dependency lock files;
• pinned or digest-referenced base images;
• explicit build commands;
• versioned pipeline definitions;
• isolated build environments;
• artifact checksums or image digests; and
• separation of build from deployment.

A deployment should reference an immutable artifact digest rather than a mutable tag such as latest. Tags are convenient labels; digests provide stronger identity.

3.3 Gates and exceptions

A gate is a condition that must be satisfied before progression. Examples include all unit tests passing, no critical vulnerability without an approved exception, or a successful staging smoke test. Exceptions must be documented with an owner, justification, expiry date, and compensating control. A permanent undocumented exception converts a control into decoration.

4. Testing Strategy and Quality Evidence

Testing is a risk-management activity, not merely a count of test cases. A balanced strategy uses different tests for different failure modes.

The test pyramid encourages many fast unit tests, fewer integration tests, and a small number of valuable end-to-end tests. It is a heuristic, not a law. A service whose principal risk is an external contract may reasonably invest heavily in contract testing.

Test results should be recorded as evidence: test name, revision, environment, result, duration, and relevant logs. A green pipeline is meaningful only when the tests address realistic risks.

4.1 Quality criteria

The project should define acceptance criteria before implementation. Examples include:

• invalid input is rejected with a documented response;
• a user without the required role cannot access an administrative operation;
• a database migration is backward compatible during rollout;
• a failed health check prevents traffic from reaching an unhealthy instance; and
• a deployment can be rolled back using a documented command or pipeline action.

Coverage percentage can identify untested code, but it does not prove correctness. Coverage should be interpreted with mutation testing, boundary cases, failure-path tests, and review.

5. Infrastructure as Code and Cloud-Native Deployment

Infrastructure as code (IaC) defines infrastructure in reviewable, repeatable files. It may provision networks, identity roles, registries, databases, clusters, monitoring, and policies. Terraform, Pulumi, CloudFormation, and native Kubernetes manifests are examples of IaC technologies.

A responsible IaC workflow is:

1. format and validate definitions;
2. run static security and policy checks;
3. generate a plan;
4. review the plan;
5. apply through controlled automation;
6. record state securely; and
7. detect and reconcile drift.

State files may contain sensitive metadata and must be protected with access control, encryption, versioning, and locking. Credentials should not be embedded in source or state where avoidable.

5.1 Cloud-native characteristics

A cloud-native design generally favours:

• stateless application instances where practical;
• externalised configuration;
• immutable images;
• health probes;
• horizontal scaling;
• declarative deployment;
• resilience to instance replacement; and
• managed services where they reduce undifferentiated operational work.

Cloud-native does not mean “use every cloud service.” The architecture should be justified by workload, cost, reliability, skill, portability, and compliance requirements.

5.2 Deployment strategies

Rolling deployment replaces instances gradually. It is resource-efficient but may expose users to mixed versions.

Blue-green deployment maintains two environments and switches traffic from the old version to the new version. Rollback is fast if the old environment remains available, but cost is higher.

Canary deployment sends a small proportion of traffic to the new version, evaluates evidence, and expands exposure if indicators remain acceptable. It requires reliable telemetry and traffic control.

A rollback is not always sufficient. Database schema changes may be irreversible, and a defective release may have modified data. Expand-and-contract migrations, backups, compatibility windows, and tested restoration procedures are therefore necessary.

6. Observability and Reliability

Observability concerns the ability to infer internal system behaviour from outputs. The principal signals are:

• Logs: timestamped records of events, preferably structured and correlated with request or trace identifiers.
• Metrics: numerical time-series measurements such as request count, error count, queue depth, CPU use, and latency.
• Traces: a distributed representation of a request as it crosses services.

Monitoring asks whether known conditions are present; observability supports investigation of conditions not anticipated in advance.

6.1 Service-level indicators and objectives

A service-level indicator (SLI) is a measured aspect of service performance. Examples are successful request proportion and latency below a threshold. A service-level objective (SLO) states the desired target over a time window.

For a request-based availability SLI:

The numerator and denominator must be defined consistently. Health-check traffic, retries, synthetic requests, and excluded maintenance periods should be specified. If 99,500 of 100,000 valid requests succeed, availability is 99.5%.

The corresponding error budget is:

For an SLO of 99.5%, the error budget is 0.5% of the measurement window. If the window contains 100,000 valid requests, the budget is 500 unsuccessful requests, assuming the request-based model and no exclusions.

For latency, a percentile such as p95 is often more informative than an arithmetic mean. p95 is the value at or below which approximately 95% of observations fall, subject to the measurement method and sample quality. Percentiles should not be averaged across services without suitable aggregation; latency histograms or raw observations are preferable.

6.2 Alert design

An alert should be actionable, attributable, and associated with a response. Alerts based only on CPU can miss user-visible failures; alerts based only on logs can be noisy. A capstone should include alerts for symptoms such as sustained error-rate increase, unavailable instances, excessive latency, and exhausted capacity, with links to dashboards and runbooks.

7. Security, Governance, and Compliance

Security must be integrated into the delivery lifecycle. The capstone should demonstrate defence in depth rather than a single scanner.

7.1 Identity and least privilege

Every human and workload should use an identifiable account or service identity. Permissions should be limited to the actions and resources required. CI systems should use short-lived federated credentials where available rather than long-lived cloud keys stored as repository secrets.

Important controls include:

• multi-factor authentication for privileged users;
• role-based access control;
• separation of development and production permissions;
• protected environments;
• auditable administrative actions; and
• periodic review of access.

7.2 Secrets and data protection

Passwords, API keys, tokens, private certificates, and encryption keys must not be committed to source control or printed in logs. They should be stored in a secret manager, injected at runtime, rotated, and scoped. Sensitive data should be classified, minimised, encrypted in transit and at rest where appropriate, and retained only as long as justified.

The project should distinguish authentication, which establishes identity, from authorisation, which determines permitted action. It should also validate input, protect dependencies, apply secure headers where relevant, and avoid exposing internal error details.

7.3 Software supply-chain security

A modern pipeline should consider the provenance of source, dependencies, build tools, base images, and artifacts. Useful practices include dependency pinning, vulnerability scanning, signed artifacts, software bills of materials (SBOMs), trusted registries, and verification before deployment.

A scanner result requires interpretation. A vulnerability may be exploitable, unreachable, mitigated by configuration, or incorrectly classified. Conversely, a clean scan does not prove absence of vulnerabilities. Risk decisions should record severity, exploitability, exposure, remediation, and acceptance authority.

Governance also includes licensing, privacy obligations, accessibility, audit retention, change approval, and incident reporting. The exact legal requirements depend on jurisdiction and sector; students must not treat a technical checklist as legal advice.

8. Metrics and Evidence-Based Evaluation

The capstone should evaluate both delivery performance and product/service outcomes. Common delivery metrics include:

• deployment frequency;
• lead time for changes;
• change failure rate; and
• time to restore service.

A useful definition of change failure rate is:

The team must define “failed,” for example as a change requiring rollback, hotfix, incident response, or emergency remediation within a specified period. If 3 of 30 production changes require rollback or urgent remediation, the rate is 10%.

Time to restore service is measured from recognition or declaration of an incident—whichever definition the organisation adopts—until service is restored to the agreed condition. Definitions must remain stable when comparing periods.

Metrics can be gamed. A high deployment frequency may result from trivial deployments; a low failure rate may reflect under-reporting. Therefore, pair quantitative indicators with qualitative evidence such as incident records, review samples, customer impact, and architecture decisions. Metrics are signals for learning, not instruments for blaming individuals.

9. Documentation and Demonstration Portfolio

A professional capstone is incomplete without documentation. The portfolio should contain:

1. Problem statement and scope: users, constraints, assumptions, and exclusions.
2. Architecture decision record: diagrams, alternatives considered, and rationale.
3. Repository guide: structure, branching, local setup, and contribution rules.
4. Pipeline guide: stages, gates, environments, secrets, and artifact handling.
5. Infrastructure guide: modules, state management, prerequisites, and destruction procedure.
6. Operations runbook: deployment, rollback, scaling, backup, restoration, and incident steps.
7. Security record: threat considerations, access model, scan results, and exceptions.
8. Observability guide: dashboards, SLIs, SLOs, alerts, and log-retention decisions.
9. Evidence index: links or screenshots labelled with revision, timestamp, environment, and result.
10. Limitations and future work: what was not proven and why.

Evidence should be reproducible. A screenshot of a dashboard without its time range, environment, and query is weak evidence. A pipeline link, immutable artifact digest, test report, and deployment record form a stronger chain.

10. Illustrative Case: Campus Appointment Service

The following is a fictional illustrative case.

Background

A university wants a web service through which students can request appointments with academic advisers. The service exposes an API, stores appointment records, and sends confirmation events. It must protect student information and remain available during registration periods.

Problem

The existing application is deployed manually on one virtual machine. Releases are infrequent, configuration differs between environments, credentials are shared, and staff cannot determine which version is running. A failed release previously caused several hours of disruption.

Analysis

The team identifies the following risks:

• an unreviewed change could expose appointment data;
• a deployment could fail because of environment drift;
• a database migration could be incompatible with the previous application version;
• a traffic spike could exhaust application capacity; and
• poor telemetry could delay diagnosis.

The team defines an SLO of 99.5% successful valid requests over a rolling 30-day period and a p95 latency objective of less than 500 ms for the appointment-search endpoint under the agreed test load. These are engineering objectives, not guarantees.

Approach and solution

The team uses a protected Git repository with short-lived branches and pull requests. Each pull request runs formatting, static analysis, unit tests, integration tests, dependency scanning, and an image build. The image is published by digest to a registry only after the pipeline succeeds.

Terraform provisions a private network, a managed database, workload identity, a container orchestration environment, and monitoring resources. Development and staging use separate accounts or projects from production. Production deployment is a canary release: 10% of traffic is directed to the new revision, and promotion requires acceptable error rate, latency, and smoke-test results.

The application uses structured logs with request identifiers, metrics for request count, errors, latency, and database connection saturation, and traces for API-to-database calls. Secrets are retrieved from a managed secret store. The database migration follows an expand-and-contract pattern: new fields are introduced compatibly, the application is updated, and obsolete fields are removed only after the old version is no longer active.

The runbook describes canary rollback, database restoration, credential rotation, and escalation. An SBOM and vulnerability report are attached to each release record.

Outcome

During a simulated release, the canary shows elevated error rates caused by an incorrect database query. The pipeline’s post-deployment checks stop promotion, traffic is returned to the previous revision, and the incident record links the failed artifact to the pull request. The team corrects the query, adds a regression test, and repeats the release.

The outcome is not merely that the corrected version works. The team demonstrates detection, controlled exposure, rollback, traceability, and learning. Its evaluation includes test reports, IaC plans, deployment records, dashboard captures, the incident timeline, and the added regression test.

Learning points

The case illustrates that DevOps capability is demonstrated through repeatability and evidence. Automation without access control is unsafe; monitoring without response procedures is incomplete; and a successful deployment without traceability is difficult to govern.

11. Challenges, Limitations, and Contemporary Developments

A capstone environment cannot reproduce every production condition. Traffic patterns may be synthetic, cloud costs may restrict scale testing, and students may not have access to regulated data. These limitations should be declared rather than hidden.

Common failure modes include:

• copying a pipeline without understanding its gates;
• using latest images and losing release identity;
• storing secrets in environment files committed to Git;
• treating coverage as proof of quality;
• creating alerts that nobody owns;
• allowing production access through personal credentials; and
• measuring deployment count without measuring user impact.

Contemporary practices include platform engineering, internal developer platforms, policy as code, OpenTelemetry-based telemetry, software supply-chain attestations, GitOps reconciliation, and AI-assisted code generation. AI can help draft tests, explain logs, or identify configuration anomalies, but generated output must be reviewed, tested, scanned, and checked for licensing, privacy, bias, and insecure patterns. AI should not receive confidential source code or credentials without explicit authorisation.

Best practice is contextual. A small organisation may choose a managed platform and a simple pipeline; a regulated organisation may require stronger segregation, approvals, evidence retention, and formal risk acceptance. The engineering obligation is to make assumptions explicit and show why controls are proportionate.

Chapter Summary

An integrated DevOps capstone demonstrates a complete, governed path from requirement to operational learning. Source control provides traceability and review; CI/CD automates repeatable validation and delivery; testing supplies risk-focused evidence; IaC makes environments reproducible; cloud-native deployment supports controlled scaling and replacement; observability reveals system behaviour; security protects identities, data, and the software supply chain; and metrics support evidence-based improvement.

A strong capstone does not depend on a particular vendor or an impressive number of tools. It defines risks, implements proportionate controls, records exceptions, tests failure paths, and presents evidence that another competent person could inspect and reproduce. The final product is therefore both a working application and a defensible engineering system.

Key Terms

DevOps: A socio-technical approach integrating development, operations, security, and feedback.

Continuous integration: Frequent integration of changes with automated validation.

Continuous delivery: Keeping software in a releasable state through automated processes.

Continuous deployment: Automatically releasing approved changes to production.

Pipeline gate: A condition that must be satisfied before a workflow proceeds.

Artifact: A versioned output of a build, such as a package or container image.

Digest: An immutable cryptographic identifier for an artifact.

Infrastructure as code: Declarative or programmatic management of infrastructure.

Drift: A difference between declared infrastructure state and actual state.

Canary deployment: Releasing a change to a small traffic proportion before wider release.

Rollback: Returning service to a previous known version or configuration.

Observability: The ability to infer internal behaviour from system outputs.

SLI: A measured indicator of a service characteristic.

SLO: A target value or range for an SLI over a defined period.

Error budget: The permitted unreliability implied by an SLO.

Least privilege: Granting only the access required for an authorised task.

SBOM: A machine-readable inventory of software components and dependencies.

Change failure rate: The proportion of production changes causing defined failure outcomes.

Runbook: Operational instructions for routine or incident activities.

Traceability: The ability to connect a requirement, change, artifact, deployment, and outcome.

Review Questions

Short Answer Questions

1. Distinguish continuous delivery from continuous deployment.
2. Why should production deployments reference artifact digests rather than only mutable tags?
3. Identify the three principal observability signals and state the main purpose of each.
4. What is the relationship between an SLO and an error budget?
5. Why is a documented exception preferable to an undocumented pipeline bypass?

Long Answer Questions

1. Design a CI/CD pipeline for a containerised API. Explain the order and purpose of its quality, security, packaging, deployment, and verification stages.
2. Explain how infrastructure as code improves repeatability and governance, and discuss the risks associated with IaC state management.
3. Compare rolling, blue-green, and canary deployment strategies. Recommend one for a student information service and justify your choice.
4. Explain how source control, artifact identity, deployment records, and observability combine to support incident investigation.
5. Discuss why delivery metrics must be interpreted with qualitative evidence and explain how metrics can be gamed.

Analytical / Application Questions

1. A team records 98,800 successful valid requests out of 100,000 in a measurement window. Calculate availability and determine whether a 99.5% SLO is met. State the assumptions in your calculation.
2. A release has 40 production changes, of which 4 require rollback or urgent remediation. Calculate the change failure rate and explain two limitations of this metric.
3. A canary release produces normal CPU usage but a sharp increase in checkout errors and p95 latency. Propose an evidence-based release decision, the immediate operational action, and two follow-up engineering actions.

Multiple Choice Questions with Answers

Question 1

Which sequence best represents a governed DevOps delivery flow?

A. Deploy, code, test, plan, observe

B. Plan, code, test, package, deploy, observe, learn

C. Test, deploy, plan, delete, code

D. Code, operate, ignore, package, deploy

Correct Answer: B. Plan, code, test, package, deploy, observe, learn

Question 2

What is the principal advantage of referencing a container image by digest?

A. It makes the image smaller

B. It provides immutable artifact identity

C. It removes the need for testing

D. It automatically encrypts application data

Correct Answer: B. It provides immutable artifact identity

Question 3

Which practice most directly supports traceability?

A. Using a mutable latest tag for every release

B. Linking a requirement to a commit, pipeline, artifact, and deployment

C. Allowing direct production changes

D. Deleting pipeline histories after deployment

Correct Answer: B. Linking a requirement to a commit, pipeline, artifact, and deployment

Question 4

Which deployment strategy initially exposes a new version to a small proportion of traffic?

A. Canary

B. Big-bang

C. Static backup

D. Manual compilation

Correct Answer: A. Canary

Question 5

If 99,500 of 100,000 valid requests succeed, what is availability?

A. 95.0%

B. 99.0%

C. 99.5%

D. 100.5%

Correct Answer: C. 99.5%

Question 6

For an SLO of 99.5%, what is the corresponding error budget under the stated model?

A. 0.05%

B. 0.5%

C. 5%

D. 99.5%

Correct Answer: B. 0.5%

Question 7

Which item should not be committed to a source repository?

A. A test file

B. A versioned pipeline definition

C. A secret API key

D. An architecture decision record

Correct Answer: C. A secret API key

Question 8

What does least privilege require?

A. Giving every developer administrator access

B. Granting only the permissions required for an authorised task

C. Sharing one account across the team

D. Disabling audit logs

Correct Answer: B. Granting only the permissions required for an authorised task

Question 9

Four of 40 production changes require rollback or urgent remediation. What is the change failure rate?

A. 4%

B. 8%

C. 10%

D. 40%

Correct Answer: C. 10%

Question 10

Which statement about test coverage is most accurate?

A. High coverage proves that software is correct

B. Coverage is irrelevant to quality

C. Coverage is evidence about exercised code but does not prove correctness

D. Coverage replaces integration and security testing

Correct Answer: C. Coverage is evidence about exercised code but does not prove correctness