Cloud-Native Development, Automation and DevOps

Introduction and context

Cloud computing has changed infrastructure from a collection of manually configured machines into an programmable service. Cloud-native development extends this idea to applications: systems are designed to exploit elasticity, automation, distributed operation, managed services and rapid change. DevOps supplies the organisational and technical practices that connect software development with operations. DevSecOps extends the same responsibility to security.

A cloud-native team does not merely move an existing application to a virtual machine. It designs repeatable delivery processes, treats infrastructure and policy as code, packages applications consistently, observes their behaviour and uses automated controls to reduce operational risk. The objective is not automation for its own sake. The objective is reliable business change: a tested change should be capable of being built, secured, deployed, observed, rolled back and audited with minimal manual intervention.

Learning objectives

By the end of this chapter, a learner should be able to:

• explain cloud-native architecture and the DevOps and DevSecOps operating models;
• use Git, APIs and scripts as foundations for automation;
• distinguish infrastructure as code, configuration management and policy as code;
• design a CI/CD pipeline with appropriate quality and security gates;
• explain containers, images, registries and orchestration;
• compare virtual machines, containers and serverless functions;
• apply deployment strategies and basic reliability measures;
• identify governance, security, ethical and regulatory obligations;
• evaluate an automation approach through a realistic case study.

1. Cloud-native principles and the DevOps model

Cloud-native is a way of designing and operating software rather than a product label. Common characteristics include:

1. Decomposable services: capabilities are separated according to business responsibility and can evolve independently where this is beneficial.
2. API-oriented integration: services communicate through explicit, versioned interfaces.
3. Immutable and reproducible delivery: a tested artifact is promoted rather than rebuilt differently in each environment.
4. Elastic operation: capacity can be adjusted in response to demand.
5. Automation and observability: provisioning, testing, deployment and operational feedback are automated as far as practical.
6. Resilience: failure of a component is expected and contained through timeouts, retries, health checks and graceful degradation.

Cloud-native does not require microservices. A well-designed modular monolith may be more appropriate for a small team or a transactionally coupled domain. Likewise, containers and Kubernetes are tools, not definitions of cloud-native maturity.

DevOps as a socio-technical system

DevOps combines culture, process and technology. Development, operations, security and business representatives share responsibility for outcomes rather than optimising isolated departmental measures. Important practices include:

• small, reviewable changes;
• version control for application and infrastructure artefacts;
• automated testing and deployment;
• shared operational metrics and incident learning;
• blameless post-incident reviews;
• clearly defined ownership and service-level objectives.

A useful distinction is between continuous delivery and continuous deployment. Continuous delivery keeps every approved change in a deployable state; release to production may require a business decision. Continuous deployment automatically releases changes that pass all required controls. The latter is not inherently superior: regulated or high-risk services may need explicit approval.

DevSecOps integrates security throughout the lifecycle. Threat modelling, dependency scanning, secret detection, least-privilege design, runtime monitoring and incident response are performed continuously rather than postponed to a final audit.

2. Version control, APIs and scripting

Git and change integrity

Git records a directed history of snapshots. A typical workflow is:

1. create or update a branch;
2. make a small change;
3. run local tests and linters;
4. commit with a meaningful message;
5. push the branch and open a pull request;
6. obtain review and automated checks;
7. merge using a documented strategy;
8. tag or otherwise identify the released artifact.

A commit is not automatically a secure change. Branch protection, signed commits where appropriate, review rules, secret scanning and retention policies improve trust. Teams should avoid storing passwords, private keys or tokens in repositories, including historical commits. If a secret is committed, removing the visible file is insufficient; the credential must be revoked or rotated and the history assessed.

Trunk-based development uses short-lived branches and frequent integration. Git-flow-style models may suit scheduled releases but can create long-running divergence. The appropriate choice depends on release risk, team size and integration frequency.

APIs as automation boundaries

An API is a defined interface through which software requests data or actions. A robust API specifies authentication, authorisation, resource representation, error behaviour, rate limits, versioning and idempotency. REST over HTTP is common, but event APIs, gRPC and provider-specific interfaces are also important.

Idempotency is particularly valuable in automation. An operation is idempotent if repeating the same request produces the same intended state as performing it once. For example, “ensure a storage bucket has encryption enabled” is safer to retry than “create another bucket”. Idempotency does not mean that every response is identical; it means that repeated execution does not create unintended additional effects.

Automation must handle timeouts, transient failures, pagination, throttling and partial completion. A script that assumes one successful response is not production automation. Retries should be bounded and use backoff; retrying a non-idempotent operation without protection can duplicate transactions.

Scripting practices

Shell, Python and PowerShell are widely used for cloud administration. Professional scripts should:

• fail explicitly and report useful errors;
• validate input and quote data safely;
• avoid embedded credentials;
• use structured output where possible;
• log actions without exposing sensitive values;
• support a dry-run mode for destructive changes;
• be tested in an isolated environment;
• be idempotent and safe to rerun;
• pin or verify dependencies.

Scripts are excellent for glue logic and one-off migration tools, but a large, stateful environment is usually better represented by a declarative infrastructure tool.

3. Infrastructure as code, configuration and policy

Infrastructure as code

Infrastructure as code (IaC) defines networks, identities, compute resources, storage, databases and related settings in machine-readable files. IaC can be declarative, describing the desired end state, or imperative, specifying a sequence of commands. Declarative tools compare desired state with observed state and calculate a plan. Examples include Terraform, OpenTofu, CloudFormation and Azure Bicep; the precise tool is less important than controlled state and review.

A professional IaC workflow separates:

• reusable modules from environment-specific values;
• source code from secrets;
• plan from apply;
• state storage from individual workstations;
• development, test and production permissions.

State must be protected because it may contain resource identifiers or sensitive values. Remote state commonly requires encryption, access control, locking and versioned recovery. Concurrent updates without locking can corrupt state or create conflicting changes.

A safe pipeline generally runs formatting, validation, static analysis and a plan. A reviewed plan is applied by a controlled identity. Manual console changes create configuration drift; either prohibit them, import and codify them, or document an emergency procedure followed by reconciliation.

Configuration management

IaC commonly creates infrastructure; configuration management establishes software and operating-system settings. Tools such as Ansible, Chef and Puppet can install packages, configure services, manage files and enforce desired settings. Image baking can move configuration earlier: a machine image is built, tested and then deployed as an immutable artifact.

Configuration management should distinguish configuration data from secrets. A database endpoint may be configuration; a database password belongs in a secret manager and should be injected at runtime or retrieved through a short-lived identity.

Policy as code

Policy as code expresses rules that can be tested and automatically enforced. Examples include:

• storage must be encrypted;
• public access is prohibited for internal databases;
• resources require owner and cost-centre tags;
• deployments must use approved image registries;
• workloads may run only in permitted regions;
• a service account may access only required resources.

Policies may be preventive, detective or corrective. A preventive admission policy can reject an unsafe deployment. A detective rule can report an existing violation. Corrective automation can remediate it, but must be designed carefully: automatic deletion or modification can cause an outage if context is misunderstood. Policy exceptions require expiry dates, owners and justification.

4. Containers and orchestration

A container packages an application and its user-space dependencies while sharing the host operating system kernel. It is therefore usually lighter and faster to start than a virtual machine, which includes a guest operating system. Containers are not a complete security boundary in every threat model; kernel vulnerabilities, excessive privileges and unsafe images remain risks.

A container image consists of immutable layers and metadata. A secure image process uses a minimal trusted base, pinned dependencies, reproducible builds, non-root execution where possible, vulnerability scanning, software bills of materials (SBOMs), signing and controlled registries. Image tags such as latest are ambiguous; digests or immutable release tags provide stronger traceability.

Orchestration concepts

An orchestrator schedules workloads and maintains desired state. Kubernetes, a widely used example, includes:

• Pod: the smallest deployable unit, usually one application container and sometimes supporting containers;
• Deployment: manages replicated, replaceable application instances;
• Service: provides stable network discovery for changing pods;
• Ingress or gateway: exposes HTTP or HTTPS routes;
• ConfigMap and Secret: supply configuration and sensitive values, subject to platform security controls;
• Namespace: provides a logical scope for resources and access policies;
• Health probes: indicate whether a process is alive and ready to receive traffic;
• Horizontal scaling: changes replica count according to metrics or rules.

A liveness probe answers whether restarting may help; a readiness probe answers whether traffic should be sent. Confusing these can produce restart loops or route traffic to an unready application. Resource requests influence scheduling, while limits constrain consumption; poorly chosen values can cause throttling, eviction or waste.

Orchestration introduces operational complexity. Teams must manage cluster upgrades, identity, network policy, admission control, storage, observability and backup. A managed control plane reduces some work but does not remove responsibility for workloads and data.

5. CI/CD pipeline design

A CI/CD pipeline is a controlled path from source change to operating service. A representative sequence is:

1. source checkout and dependency restoration;
2. formatting, linting and static analysis;
3. unit tests;
4. build of a versioned artifact;
5. dependency, secret and software-composition scans;
6. image build and signing if applicable;
7. integration and contract tests;
8. infrastructure validation and policy checks;
9. deployment to an ephemeral or test environment;
10. acceptance, performance and security tests;
11. approval or automated release decision;
12. progressive production deployment;
13. monitoring, verification and rollback or promotion.

The pipeline itself is production infrastructure. It needs access control, isolated runners, protected variables, audit logs and dependency maintenance. Build environments should be reproducible and preferably ephemeral. A pipeline identity should have only the permissions needed for its stage; a test job should not possess unrestricted production deletion rights.

Deployment strategies

• Recreate: stop the old version and start the new one. It is simple but causes downtime.
• Rolling: replace instances gradually. It saves capacity but requires compatibility between versions.
• Blue-green: maintain two environments and switch traffic. Rollback is rapid, but duplicate capacity may be expensive.
• Canary: expose a small proportion of users or traffic to the new version and expand if indicators remain healthy.
• Feature flags: deploy code separately from enabling a feature. Flags need ownership, testing and removal dates.

Database changes require special care. A backward-compatible expand-and-contract migration can add a new field, deploy code that supports both forms, migrate data, switch reads and later remove the old field. Deploying an irreversible schema change before all application instances are compatible can make rollback impossible.

Measuring delivery and reliability

Useful measures include deployment frequency, lead time for changes, change failure rate and time to restore service. These metrics are most useful as a balanced set, not as targets to manipulate. Reliability is commonly represented through service-level indicators (SLIs), objectives (SLOs) and agreements (SLAs).

For availability over an observation period:

Both time quantities must use the same unit, and the organisation must define exclusions such as planned maintenance. A monthly target of 99.9% over a 30-day month permits approximately 43.2 minutes of unavailable time because 30 × 24 × 60 × 0.001 = 43.2 minutes. This is a planning calculation, not proof that users experienced equal service quality.

An error budget is the permitted unreliability implied by an SLO. If the availability objective is 99.9%, the error-budget fraction is 0.1%. Teams can use the budget to balance release speed and reliability: repeated failures may justify slowing risky changes and investing in resilience.

6. Serverless computing

Serverless computing abstracts server provisioning and often charges according to invocations, execution duration, requests or allocated resources. Functions as a service (FaaS) are typically short-lived, event-driven and stateless between invocations. Other serverless services include managed databases, queues, workflow engines and object storage.

A function may be triggered by an HTTP request, message, schedule, object upload or database event. Good designs externalise state, make handlers idempotent, use dead-letter handling for failed messages and apply explicit timeouts. Cold starts can increase latency when a new execution environment is created; provisioned capacity or smaller dependencies may reduce this at additional cost.

Serverless is not “without servers”; responsibility for servers shifts to the provider. The customer still designs identity, permissions, data protection, observability, dependency management, concurrency limits and failure handling. It is attractive for irregular workloads and rapid development, but long-running processes, specialised networking, predictable high utilisation or strict portability requirements may favour other models. Vendor-specific triggers can create lock-in, so APIs, events and data formats should be documented.

7. DevSecOps, governance and responsible automation

Security controls should be proportionate to risk and integrated into normal work. Key controls include:

• central identity with multifactor authentication;
• least privilege and short-lived credentials;
• encryption in transit and at rest, with managed key rotation;
• network segmentation and egress control;
• dependency and image scanning;
• signed artifacts and provenance records;
• centralised, tamper-resistant logs;
• backup, recovery and tested incident procedures;
• vulnerability remediation based on exploitability and business impact.

Compliance is not equivalent to security. Regulations may impose requirements concerning personal data, retention, residency, breach notification, accessibility and auditability. Data minimisation, purpose limitation and controlled retention reduce privacy risk. Logs can themselves contain personal or confidential information and must be governed.

Automation also creates ethical risks. An overly broad remediation rule may delete a legitimate resource; an opaque policy may exclude a team without a fair appeal process; cost optimisation may reduce availability for users with accessibility needs. Human approval should remain for high-impact, destructive or legally sensitive actions. Every automated decision should have an owner, rationale, audit trail and rollback or appeal mechanism.

Artificial intelligence can assist with log summarisation, test generation, anomaly detection and infrastructure suggestions. It can also generate insecure code, expose confidential prompts or produce unverifiable changes. AI-generated infrastructure must pass the same review, tests, policy checks and provenance requirements as human-generated code.

8. Applied case: Northstar Health appointment platform

The following is an illustrative case, not a report of a real organisation.

Background and problem

Northstar Health operates an appointment platform for clinics. Its manually configured virtual machines are difficult to reproduce. Releases occur every six weeks, emergency changes are common, and a failed deployment once exposed an internal administration endpoint. The platform handles personal data, so auditability and access control are essential. Demand rises sharply during seasonal campaigns.

Analysis

The principal risks are configuration drift, excessive production privileges, weak release verification, unprotected secrets and an inability to scale selected components independently. A complete microservices rewrite would introduce unnecessary risk. The team therefore chooses a modular application first, with asynchronous workers separated where scaling requires it.

Approach and solution

The team establishes a Git repository with protected main branches and mandatory reviews. IaC defines networks, private database access, workload identities, logging and an encrypted state store. A policy engine rejects public database exposure, missing ownership tags and unsigned images.

The build pipeline runs tests, dependency scans, secret detection and SBOM generation. It builds an immutable image as a non-root user, signs it and stores it in a private registry. A deployment controller applies the reviewed manifest to a managed Kubernetes service. Readiness probes prevent traffic reaching an unready release; resource requests and limits are measured and tuned.

The team uses an expand-and-contract database migration process and a canary release: 5% of traffic is routed to the new version, then increased only if latency, error rate and appointment-completion indicators remain within SLO thresholds. A separate serverless function processes non-critical reminder events from a queue. Secrets are retrieved through workload identity rather than committed to Git.

Security and governance are built into the workflow. Production deployment requires a protected environment, two-person approval for high-risk changes and an audit record. Logs are filtered to avoid unnecessary patient information. A disaster-recovery exercise verifies restoration of encrypted backups and re-creation of infrastructure from code.

Outcome

The platform becomes reproducible and releases become smaller and more frequent. The team can identify which artifact is running, detect policy violations before deployment and roll back the application without reversing a database change. The case does not eliminate incidents: instead, it reduces the probability and impact of incidents and improves recovery.

Learning points

1. Automation is valuable only when its state, permissions and failure modes are controlled.
2. A modular architecture can provide cloud-native benefits without an immediate microservices rewrite.
3. Security, compliance and reliability requirements must shape the pipeline from its first stage.
4. Progressive delivery and observability make release decisions evidence-based.
5. Infrastructure code must be reviewed and tested like application code.

Challenges and limitations

Cloud-native adoption can increase complexity, cloud expenditure and dependence on scarce skills. Distributed systems introduce network failures, eventual consistency, tracing difficulty and more complicated testing. Autoscaling may amplify a faulty workload or generate unexpected cost. Managed services reduce undifferentiated operations but can limit portability and visibility.

Automation can also reproduce errors at high speed. A faulty IaC module may alter many environments; a compromised pipeline can become a supply-chain attack path. Controls therefore include small scopes, approvals for destructive actions, policy tests, immutable artifacts, backups and rehearsed recovery. Teams should adopt the simplest architecture that meets reliability, security and delivery needs, and periodically remove obsolete services, feature flags, credentials and policies.

Best-practice checklist

• Keep application, infrastructure, policy and pipeline definitions under version control.
• Make builds reproducible and artifacts immutable and traceable.
• Separate plan, approval and apply permissions.
• Use short-lived credentials and a secret manager.
• Test scripts for idempotency, failure and safe retry behaviour.
• Scan dependencies and images, generate SBOMs and verify provenance.
• Define SLIs and SLOs before selecting scaling and release rules.
• Prefer progressive delivery for high-risk changes.
• Monitor cost, security, performance and user outcomes together.
• Review automation after incidents and remove unnecessary complexity.

Chapter Summary

Cloud-native development combines programmable infrastructure, APIs, automation, observability and resilient application design. Git provides traceable change control; scripts and APIs provide automation interfaces; IaC defines repeatable infrastructure; configuration management establishes desired software settings; and policy as code enforces security, compliance and operational rules. Containers package consistent artifacts, while orchestration schedules, exposes, scales and heals workloads at the cost of additional complexity. CI/CD pipelines convert reviewed source changes into tested and controlled releases, using strategies such as rolling, blue-green and canary deployment. Serverless computing abstracts more infrastructure and suits event-driven workloads but retains responsibilities for identity, data, observability and failure handling. DevSecOps embeds security and governance throughout the lifecycle. Effective practice balances speed with reliability, least privilege, auditability, cost control and responsible human oversight.

Key Terms

Cloud-native: A design and operating approach that uses automation, elastic resources, APIs, resilience and observable services.

DevOps: A socio-technical approach integrating development, operations and shared responsibility for delivery and reliability.

DevSecOps: DevOps with security activities and controls integrated throughout the lifecycle.

Git: A distributed version-control system that records project snapshots and their history.

API: A defined interface through which software requests data or actions.

Idempotency: The property that repeating an operation produces the same intended state as performing it once.

Infrastructure as code: Machine-readable definition and management of infrastructure resources.

Declarative configuration: A description of desired state rather than a sequence of operational commands.

Configuration management: Automated establishment and maintenance of software and system settings.

Policy as code: Machine-readable rules that evaluate or enforce governance and security requirements.

Container: An isolated process environment that packages an application and user-space dependencies.

Image: An immutable, versioned package used to create containers.

Orchestration: Automated scheduling, networking, scaling and lifecycle management of workloads.

Continuous integration: Frequent integration of changes with automated build and test activities.

Continuous delivery: Keeping tested changes in a deployable state.

Continuous deployment: Automatically releasing approved changes that pass required controls.

Serverless: A cloud model in which the provider manages server capacity and the customer consumes event-driven or managed services.

SLO: A target value for a service-level indicator over a defined period.

Error budget: The unreliability permitted by an SLO.

Software bill of materials: An inventory of components and dependencies in a software artifact.

Review Questions

Short Answer Questions

1. Distinguish continuous delivery from continuous deployment.
2. Why is idempotency important in cloud automation?
3. State two differences between a container and a virtual machine.
4. What is the purpose of a readiness probe?
5. Explain the relationship between an SLO and an error budget.

Long Answer Questions

1. Explain how Git, IaC, configuration management and policy as code combine to create a controlled infrastructure lifecycle.
2. Describe the stages of a secure CI/CD pipeline and justify the position of at least four security controls.
3. Compare rolling, blue-green and canary deployment strategies, including their operational risks.
4. Evaluate the benefits and limitations of serverless computing for an event-driven business application.
5. Explain how DevSecOps changes organisational responsibility, technical practice and governance.

Analytical / Application Questions

1. A team wants to deploy a new API and database schema in one irreversible production operation. Analyse the risks and propose a safer migration and release sequence.
2. An automated rule deletes every untagged cloud resource each night. Assess this design from reliability, governance and ethical perspectives, then redesign it.
3. Design a high-level CI/CD and runtime approach for a personal-data application that requires auditability, canary releases and rapid rollback. Identify assumptions and trade-offs.

Multiple Choice Questions with Answers

Question 1

Which statement best describes infrastructure as code?

A. A manual checklist for configuring servers

B. Machine-readable definitions used to create and manage infrastructure

C. A password storage method for cloud accounts

D. A monitoring dashboard for application users

Correct Answer: B. Machine-readable definitions used to create and manage infrastructure

Question 2

Why is an idempotent operation valuable in automation?

A. It always executes faster than a non-idempotent operation

B. It eliminates the need for authentication

C. It can be safely repeated without creating unintended additional effects

D. It prevents all network failures

Correct Answer: C. It can be safely repeated without creating unintended additional effects

Question 3

What is the primary purpose of a readiness probe in an orchestrated workload?

A. To decide whether a workload should receive traffic

B. To encrypt the container image

C. To create a new database schema

D. To replace source-code review

Correct Answer: A. To decide whether a workload should receive traffic

Question 4

Which practice provides the strongest artifact traceability?

A. Using the mutable tag latest for every release

B. Building a different artifact in each environment

C. Using immutable identifiers such as a digest and recording provenance

D. Copying binaries manually between servers

Correct Answer: C. Using immutable identifiers such as a digest and recording provenance

Question 5

A service has a 99.9% availability objective over a 30-day month. Approximately how much unavailability does the corresponding budget permit?

A. 4.32 minutes

B. 43.2 minutes

C. 432 minutes

D. 1,440 minutes

Correct Answer: B. 43.2 minutes

Question 6

Which deployment strategy initially exposes a new version to a small proportion of production traffic?

A. Recreate

B. Canary

C. Big-bang migration

D. Cold backup

Correct Answer: B. Canary

Question 7

Which item should generally be stored in a secret manager rather than a Git repository?

A. A documented API schema

B. A source-code license

C. A database password

D. A unit test name

Correct Answer: C. A database password

Question 8

What is a major responsibility that remains with the customer when using serverless functions?

A. Manufacturing physical servers

B. Managing the provider’s data-centre power system

C. Designing identity, permissions, observability and failure handling

D. Installing a guest operating system on every invocation

Correct Answer: C. Designing identity, permissions, observability and failure handling

Question 9

What does policy as code enable?

A. Rules to be tested and automatically enforced against resources or deployments

B. Applications to run without any identity controls

C. Every cloud resource to become publicly accessible

D. Manual configuration without audit records

Correct Answer: A. Rules to be tested and automatically enforced against resources or deployments

Question 10

Which statement best reflects DevSecOps?

A. Security is performed only after production deployment

B. Security responsibility belongs exclusively to a separate audit team

C. Security controls and activities are integrated throughout development and operations

D. Security is replaced by automated scaling

Correct Answer: C. Security controls and activities are integrated throughout development and operations