Deployment, DevOps and Operational Reliability

Introduction

A web application is not complete when its code runs on a developer’s laptop. It becomes a useful information system only when it can be released safely, configured correctly, protected from misuse, observed during operation, recovered after failure, and maintained at an acceptable cost. Deployment is therefore both a technical activity and an operational discipline.

This chapter examines the practices required to move a full-stack application from development into reliable production operation. It covers environment configuration, version control, continuous integration and continuous delivery, containers and managed hosting, databases, HTTPS, secrets, logging, monitoring, incident response, rollback, backups, availability, and cloud-cost control.

DevOps is not simply a collection of tools. It is a way of organising collaboration between development and operations so that software can be delivered frequently while preserving security, quality, and reliability. Operational reliability means that the application performs its intended function consistently, recovers from faults, and provides useful evidence when something goes wrong.

Learning Objectives

By the end of this chapter, a learner should be able to:

1. Distinguish development, testing, staging, and production environments.
2. Design a basic CI/CD pipeline for a full-stack web application.
3. Explain the roles of containers, managed hosting, databases, and reverse proxies.
4. Configure HTTPS, environment variables, and secrets safely.
5. Implement useful logs, metrics, health checks, and alerts.
6. Apply rollback, backup, restore, and incident-response procedures.
7. Evaluate availability, recovery objectives, and operational risk.
8. Identify practical methods for controlling cloud expenditure.

1. From Source Code to Running Service

Deployment is the controlled process of making a software version available in a target environment. A release includes more than application source code. It may include compiled assets, database migrations, configuration, container images, infrastructure definitions, and documentation.

A typical web application contains:

• a browser-based frontend;
• an application server or API;
• a relational or non-relational database;
• file or object storage;
• background workers or scheduled jobs;
• a reverse proxy or load balancer; and
• external services such as payment, email, or authentication providers.

The reliability of the whole service is limited by interactions among these components. A fast API cannot compensate for a database that is unavailable. A secure server cannot protect credentials accidentally committed to a public repository. Deployment must therefore be considered as a system-level process.

Environments

An environment is a complete execution context, including software, configuration, data, network access, and operational controls. Common environments are:

The application should behave consistently across environments, but the environments must not be identical in every respect. For example, production may use a managed database and multiple application instances, while development uses a local database. The principle of configuration separation prevents accidental use of production credentials or data during development.

A useful design is to keep the same application artifact across environments and change only approved configuration values. Rebuilding different source versions for staging and production creates uncertainty: the version tested may not be the version released.

2. Configuration, Environment Variables, and Secrets

Configuration includes values that vary between environments, such as a database host, port, API endpoint, feature flag, or logging level. An environment variable is a named value supplied to a process by its execution environment. For example, an application may read DATABASE_URL rather than containing a database address in source code.

Environment variables are useful, but they are not automatically secret. A value can be exposed through process listings, diagnostic output, crash reports, deployment logs, or an accidentally public hosting interface. Passwords, signing keys, API tokens, and private certificates should be stored in a secrets manager or protected deployment system.

Good practice includes:

• commit a safe .env.example, never a real .env file;
• use separate credentials for each environment;
• grant services only the permissions they require;
• rotate credentials periodically and after suspected exposure;
• avoid printing secrets in logs;
• validate required configuration at startup; and
• document the owner, purpose, and rotation procedure for each secret.

Configuration validation should fail early. If a production API starts without a database URL, it is safer for deployment to fail than for the application to start in a partially functional state.

3. Version Control and Release Design

A release should be traceable to a source-control commit, build output, and deployment record. Tags, immutable artifact identifiers, and release notes provide this traceability. A production operator should be able to answer: What is running? When was it deployed? Who approved it? What changed? How can it be restored?

A practical branching model for a diploma-level project is a protected main branch with short-lived feature branches. Changes are reviewed through pull requests, automatically tested, and merged only when required checks succeed. Long-lived branches increase the likelihood of integration conflicts and make it difficult to know what has actually been tested.

Database migrations require special care. A migration changes the structure or constraints of persistent data. A safe migration should be:

1. versioned and repeatable in a controlled sequence;
2. tested against a copy or representative instance;
3. compatible with the application version during a transition where possible; and
4. accompanied by a backup or recovery plan.

A destructive operation, such as dropping a column, should generally be separated into stages: first stop writing to the old column, then deploy code that no longer depends on it, and only later remove it after verification.

4. Continuous Integration and Continuous Delivery

Continuous integration (CI) automatically builds and verifies changes whenever they are proposed or merged. Continuous delivery means that validated software is always in a releasable state; continuous deployment goes further by automatically releasing approved changes to production.

A basic pipeline is:

Developer commit
      ↓
Static analysis and dependency checks
      ↓
Unit and integration tests
      ↓
Build application artifact or container image
      ↓
Security and configuration checks
      ↓
Deploy to staging
      ↓
Smoke tests and approval
      ↓
Deploy to production
      ↓
Health checks and monitoring

Pipeline stages should be fast enough to encourage use, but complete enough to detect important defects. Typical checks include formatting, linting, type checking, unit tests, API tests, database migration tests, dependency vulnerability scanning, and build reproducibility.

A pipeline is not a substitute for judgment. Automated tests can miss poor usability, incorrect business assumptions, or failures caused by external providers. The delivery process should define approval responsibilities and a clear procedure for emergency changes.

Deployment strategies

A rolling deployment replaces application instances gradually. It is efficient but requires compatibility between old and new versions. A blue-green deployment maintains two production environments: traffic is directed to the current blue version, while the green version is validated before traffic switches. Rollback can be rapid because traffic can be returned to blue.

A canary deployment sends a small proportion of traffic to the new version and compares error rates, latency, and business indicators. It reduces exposure but requires routing controls and meaningful monitoring. For a small student project, a controlled rolling release with a tested rollback procedure is usually sufficient.

5. Containers and Managed Hosting

A container packages an application and its dependencies in an isolated user-space environment that shares the host operating system kernel. It is not a complete virtual machine. A container image is an immutable package from which containers are created; a container is a running instance of that image.

Containers improve portability and consistency, but they do not automatically provide security or reliability. Images should be small, built from maintained base images, scanned for known vulnerabilities, and run as a non-root user when possible. Secrets should not be baked into an image because image layers may persist in registries.

A typical containerised service exposes a port, reads configuration from the environment, writes logs to standard output, and stores persistent data outside the container. Containers should be treated as replaceable. A database’s durable files should not depend on the lifetime of an application container.

Managed hosting abstracts much of the server administration. The provider may operate the operating system, TLS certificates, scaling controls, backups, and deployment interface. This reduces operational workload but creates provider dependency, recurring cost, and responsibility for understanding the provider’s limits. Managed services still require correct application configuration, access control, monitoring, and data governance.

6. Databases in Production

Production database design involves correctness, durability, performance, and recoverability. A database should not be exposed directly to the public internet unless there is a compelling, well-controlled reason. Application access should use encrypted connections, restricted network rules, and a dedicated account with minimum required privileges.

Connection pooling prevents every request from creating a new database connection. However, the pool size must be compatible with the database’s connection limit and the number of application instances. Excessive pools can exhaust the database even when application traffic is moderate.

Database backups have two distinct properties:

• Recovery Point Objective (RPO): the maximum acceptable amount of data loss measured in time.
• Recovery Time Objective (RTO): the maximum acceptable time to restore service after a disruption.

For example, an RPO of 15 minutes means that losing up to 15 minutes of recent changes may be acceptable. An RTO of 2 hours means the service should be restored within two hours. These are business requirements, not merely technical settings.

A backup is not proven until restoration has been tested. Backups should be encrypted, access-controlled, retained according to policy, and stored separately enough to reduce the risk that one incident destroys both primary data and backups. Point-in-time recovery may reduce data loss, while periodic full backups simplify restoration.

7. HTTPS and Application Security

HTTPS uses TLS to protect communication between a client and server. It provides encryption, server authentication through certificates, and integrity protection against undetected modification in transit. All production web applications should redirect HTTP to HTTPS and use certificates managed through a trusted certificate authority.

TLS does not make an insecure application secure. It does not prevent SQL injection, broken access control, malicious uploads, weak passwords, or compromised servers. Application security remains necessary.

Operational security controls include:

• secure cookie flags such as Secure, HttpOnly, and an appropriate SameSite policy;
• strong authentication and session expiration;
• input validation and parameterised database queries;
• dependency updates and vulnerability review;
• restricted administrative interfaces;
• rate limiting where abuse is possible; and
• security headers appropriate to the application.

Logs must avoid personal data and credentials unless there is a documented, lawful, and protected reason to retain them. Data protection requirements may include consent, purpose limitation, retention limits, access rights, and breach notification. The exact obligations depend on jurisdiction and the nature of the data.

8. Observability: Logs, Metrics, and Traces

Observability is the ability to understand a system’s internal state from its external outputs. The three commonly discussed signals are logs, metrics, and traces.

A log is a timestamped event record. Good structured logs include an event name, severity, service, request or correlation identifier, and relevant non-sensitive context. They should support diagnosis rather than merely record messages such as “something failed.”

Metrics are numerical measurements collected over time. Useful web-service metrics include request count, error count, response latency, active users, queue depth, database connection usage, and resource consumption. Averages alone can hide slow requests; percentiles such as p95 or p99 often provide a better view of user experience. A percentile is the value below which a specified percentage of observations fall.

A trace follows a request across services. It is particularly useful when a slow response involves an API, database, cache, and external provider. Correlation identifiers allow logs from these components to be connected.

Health endpoints should distinguish between:

• liveness: whether the process is running; and
• readiness: whether the instance is prepared to receive traffic, including required dependencies.

A liveness check that tests every dependency may cause restart loops during a temporary database outage. The design of health checks must therefore reflect their purpose.

9. Reliability Metrics and Service Objectives

Availability describes the proportion of time a service is usable according to a defined measurement. If total scheduled time is T and unavailable time is D, then:

T and D must use the same time unit, and the measurement window and definition of “unavailable” must be specified. If a service is scheduled for 30 days, or 43,200 minutes, and is unavailable for 43.2 minutes:

This calculation does not prove that every user experienced the same availability. Monitoring scope, planned maintenance, regional differences, and partial failures must be considered.

A Service Level Indicator (SLI) is a measured aspect of service performance, such as successful request proportion. A Service Level Objective (SLO) is the target for that indicator, such as 99.9% successful requests during a month. A Service Level Agreement (SLA) is a formal commitment, often including remedies, between a provider and customer.

Error budgets express the amount of unreliability permitted by an SLO. For an availability target A over a period T, the permitted unavailability is:

The calculation assumes that A is expressed as a fraction. For 99.9% availability over 43,200 minutes, the budget is 0.001 × 43,200 = 43.2 minutes. Error budgets support balanced decisions: if a team has exhausted its budget, it should prioritise reliability work rather than increasing release risk.

10. Incident Response and Rollback

An incident is an event that materially affects service quality, security, data integrity, or users. Incident response should be designed before an emergency. A basic response process is:

1. Detect and acknowledge the event.
2. Assess impact and assign an incident lead.
3. Contain the problem, prioritising user and data safety.
4. Communicate accurately to affected stakeholders.
5. Restore service through rollback, repair, or failover.
6. Verify recovery using technical and business signals.
7. Conduct a blameless review and track corrective actions.

A rollback returns software or configuration to a known working state. It is straightforward when artifacts are immutable and releases are versioned. Database rollback can be difficult because data transformations may not be reversible. A forward fix or restore from backup may be safer than attempting to reverse a destructive migration.

Incident communication should distinguish facts, hypotheses, impact, and next actions. A post-incident review should ask which controls failed and how the system can be improved, rather than focusing on individual blame. Blamelessness does not mean absence of accountability; it means examining system conditions so that learning produces durable change.

11. Availability, Scaling, and Cost Control

Availability is improved through redundancy, graceful degradation, health-based routing, capacity planning, and removal of single points of failure. However, redundancy increases cost and complexity. A small application may not need multi-region deployment; a critical financial service may require it.

Scaling can be vertical, by increasing the resources of one instance, or horizontal, by adding instances. Horizontal scaling requires stateless application design or an external session store. Background jobs should be idempotent where possible: repeating the same job should not create duplicate payments or records.

Cloud cost control is an engineering responsibility. Practical measures include:

• selecting resources according to measured workload rather than guesswork;
• removing unused disks, IP addresses, databases, and test environments;
• applying budgets and spending alerts;
• using autoscaling with sensible minimum and maximum limits;
• setting log retention periods;
• selecting storage tiers according to access needs; and
• tagging resources by project, owner, and environment.

Cost optimisation must not undermine security or reliability. Disabling backups to reduce cost is not optimisation if a later failure causes unacceptable data loss. The correct decision compares cost with business impact and risk.

12. Illustrative Case Study: Deploying a Student Event Platform

Background

A college team has built an event-registration platform using a JavaScript frontend, a REST API, and a relational database. Users can browse events, register, and receive confirmation emails. The prototype works locally but has no deployment pipeline, no backups, and no reliable method of identifying failures.

Problem

The team initially copies source files manually to a virtual server. The production database password appears in a configuration file. A change to the registration table is applied manually and causes an error in the live application. Logs are plain text and contain user email addresses. When the server becomes unavailable, the team cannot determine whether the problem is the application, database, network, or email provider.

Analysis

The main risks are uncontrolled release, exposed credentials, untested schema changes, inadequate observability, and unverified recovery. The platform is not necessarily required to provide global, multi-region availability, but registration data must not be silently lost and service should be restored promptly during college working hours.

The team defines an initial RPO of 15 minutes and RTO of 2 hours. It chooses a managed relational database with automated point-in-time recovery, while retaining periodic encrypted exports for independent recovery. It defines an SLO for successful registration requests and excludes only documented maintenance from the service measurement.

Approach and Solution

The team adopts a protected main branch and pull-request review. The CI pipeline runs tests, static analysis, dependency checks, and a build. The build produces an immutable container image identified by its commit hash. The same image is deployed to staging and then production.

Production secrets are moved to a secrets manager. The application receives only the database credentials and email token it needs. Database access is restricted to the application network, and the database connection uses TLS. HTTPS is enabled at the managed reverse proxy, with automatic certificate renewal.

The application writes structured logs without email addresses or passwords. Each request receives a correlation identifier. Monitoring records request success rate, latency, database connection usage, and container health. Alerts are based on sustained conditions rather than a single transient error.

Migrations are tested against a staging copy. A registration-table change follows an expand-and-contract pattern: the new structure is added compatibly, code is deployed, data is verified, and obsolete fields are removed only after a later release. A documented rollback returns the application image to the previous version, while data restoration procedures are maintained separately.

Outcome

Deployments become repeatable and auditable. A failed release is detected by staging tests and does not reach users. When a simulated database outage occurs, the team identifies the dependency through metrics and restores service within the target time. A restore exercise confirms that backups are usable rather than merely present. Cloud spending is controlled through resource tags, budgets, and scheduled shutdown of non-production environments.

Learning Points

The case illustrates that reliability is created by coordinated practices, not by a single platform feature. Immutable artifacts, automated verification, least-privilege secrets, tested backups, useful telemetry, and rehearsed response procedures reduce both failure probability and recovery time. The team also avoids overengineering: it selects controls proportional to the platform’s educational purpose and operational risk.

13. Contemporary Developments and Professional Practice

Platform engineering increasingly provides reusable deployment templates, approved infrastructure modules, and standard observability. Infrastructure as Code represents servers, networks, policies, and services in version-controlled definitions, improving repeatability and reviewability. Policy as Code can prevent deployment of resources that violate security or cost rules.

Artificial intelligence can assist with log summarisation, anomaly detection, test generation, and incident investigation. It must not be treated as an autonomous authority. Generated explanations may be incorrect, sensitive logs may be exposed to an external service, and automated remediation can amplify an error. Human approval, access controls, audit trails, and data minimisation remain essential.

Software supply-chain security is also increasingly important. Teams should know where dependencies originate, verify package integrity where practical, scan images, minimise privileges, and maintain a software bill of materials when required. Reliability and security are connected: an untrusted dependency can create both a breach and an outage.

Challenges and Limitations

No deployment model eliminates failure. Automated pipelines can automate a bad decision, monitoring can generate alert fatigue, and a managed service can fail outside the application team’s control. Staging environments may not reproduce production scale or data characteristics. Backups may be corrupted, credentials may be leaked, and cost estimates may be wrong under unexpected traffic.

Reliability measures also involve trade-offs. Strong consistency may increase latency; extensive logs may increase cost and privacy risk; redundancy may improve availability but increase complexity; and frequent deployment may accelerate learning while increasing change risk. Professional practice requires explicit assumptions, measured results, and proportional controls.

Chapter Summary

Deployment is the controlled movement of a tested software artifact into an operating environment. Reliable web applications separate configuration from code, protect secrets, use version control, and automate repeatable CI/CD checks. Containers and managed hosting improve consistency and reduce administration, but they do not remove responsibility for security, data, monitoring, and recovery.

Production databases require restricted access, encryption, migration discipline, tested backups, and clearly defined RPO and RTO values. HTTPS protects communication but must be combined with secure application design. Observability combines structured logs, metrics, traces, health checks, and meaningful alerts. Availability and error-budget calculations help teams express reliability targets quantitatively.

Incident response depends on preparation, containment, accurate communication, rollback or repair, verification, and blameless learning. Availability, scalability, and cloud cost must be balanced rather than optimised independently. The central professional principle is simple: design the release and operation of a system with the same care used to design its features.

Key Terms

Environment: A complete execution context containing software, configuration, data, and operational controls.

Deployment: The controlled process of making a software version available in a target environment.

DevOps: A collaborative approach integrating software development, delivery, operations, and feedback.

Continuous integration: Automatic building and verification of proposed code changes.

Continuous delivery: Keeping validated software in a releasable state.

Continuous deployment: Automatically releasing approved changes to production.

Container: An isolated running process packaged with application dependencies.

Container image: An immutable package used to create containers.

Environment variable: A named configuration value supplied to a running process.

Secret: Sensitive information such as a password, token, or private key.

Migration: A versioned change to a database schema or data structure.

HTTPS: HTTP protected by TLS encryption, authentication, and integrity controls.

Observability: The ability to infer internal system state from external outputs.

Log: A timestamped record of an event.

Metric: A numerical measurement collected over time.

Trace: A record following a request across components.

RPO: Maximum acceptable data loss measured as time.

RTO: Maximum acceptable restoration time after disruption.

Rollback: Returning software or configuration to a known working state.

Error budget: The permitted unreliability implied by a service objective.

Review Questions

Short Answer Questions

1. Why should the same application artifact be promoted from staging to production?
2. Distinguish an environment variable from a secret.
3. What is the difference between liveness and readiness checks?
4. Define RPO and RTO.
5. Why must database restoration be tested?

Long Answer Questions

1. Explain how a CI/CD pipeline can reduce deployment risk while recognising its limitations.
2. Discuss the security and operational considerations of deploying a containerised web application.
3. Explain how logs, metrics, and traces complement one another during incident diagnosis.
4. Describe a safe approach to database migrations when old and new application versions may run together.
5. Analyse the relationship between availability, error budgets, redundancy, and cloud cost.

Analytical / Application Questions

1. A service operates for 43,200 minutes in a month and experiences 86.4 minutes of unavailability. Calculate its availability and determine whether it meets a 99.9% target. Explain your reasoning.
2. Design a deployment and recovery plan for an online examination application that requires an RPO of 15 minutes and an RTO of 1 hour. Identify backups, monitoring, rollback, and communication procedures.
3. A new release increases p95 latency but does not increase the average latency. Propose an investigation using metrics, logs, traces, and staged rollout controls. Explain which evidence would justify rollback.

Multiple Choice Questions with Answers

Question 1

Which practice most directly prevents production credentials from being committed to source control?

A. Placing credentials in a protected secrets manager

B. Using a longer function name

C. Increasing the container CPU limit

D. Adding more unit-test assertions

Correct Answer: A. Placing credentials in a protected secrets manager

Question 2

What is the primary purpose of continuous integration?

A. To replace all production monitoring

B. To automatically build and verify proposed code changes

C. To store database backups in application containers

D. To eliminate the need for version control

Correct Answer: B. To automatically build and verify proposed code changes

Question 3

A service operates for 43,200 minutes and is unavailable for 43.2 minutes. What is its availability?

A. 90%

B. 99%

C. 99.9%

D. 99.99%

Correct Answer: C. 99.9%

Question 4

Which statement best describes RPO?

A. The maximum acceptable time required to restore service

B. The maximum acceptable amount of data loss measured in time

C. The percentage of successful requests

D. The maximum number of application instances

Correct Answer: B. The maximum acceptable amount of data loss measured in time

Question 5

Why should persistent database data normally be stored outside an application container?

A. Containers cannot expose network ports

B. Containers are replaceable and may have a short lifecycle

C. External storage always costs nothing

D. Databases cannot use encryption in containers

Correct Answer: B. Containers are replaceable and may have a short lifecycle

Question 6

Which observability signal follows one request across multiple services?

A. A trace

B. A static code analysis report

C. A firewall rule

D. A deployment tag

Correct Answer: A. A trace

Question 7

What is the main advantage of a blue-green deployment?

A. It removes the need for testing

B. It allows traffic to switch between two prepared environments

C. It guarantees that database migrations are reversible

D. It prevents all infrastructure costs

Correct Answer: B. It allows traffic to switch between two prepared environments

Question 8

Which action is most appropriate immediately after detecting a significant production incident?

A. Delete all logs

B. Assign response ownership and assess impact

C. Change unrelated application features

D. Disable all backups

Correct Answer: B. Assign response ownership and assess impact

Question 9

For a 99.9% availability target over 43,200 minutes, what is the error budget?

A. 4.32 minutes

B. 43.2 minutes

C. 432 minutes

D. 4,320 minutes

Correct Answer: B. 43.2 minutes

Question 10

Which cloud-cost control is generally appropriate?

A. Remove all monitoring to reduce charges

B. Use budgets, spending alerts, and remove unused resources

C. Disable encryption on production data

D. Store unlimited logs forever

Correct Answer: B. Use budgets, spending alerts, and remove unused resources