DevOps, Cloud Deployment and Observability

Introduction

Modern web systems are not completed when source code is compiled. They must be packaged, tested, deployed, monitored, secured, updated, and recovered repeatedly. DevOps is the organisational and technical approach that integrates software development, operations, security, and business objectives throughout the delivery lifecycle. Cloud platforms extend this lifecycle by providing programmable computing, storage, networking, managed databases, and global delivery services. Observability supplies evidence about what a running system is doing and why.

For postgraduate IT engineering students, the important issue is not learning a particular vendor tool. It is learning how to design a reliable delivery system: how a change travels from a developer’s workstation to production, how infrastructure is represented as code, how deployment risk is controlled, and how operational evidence supports engineering decisions.

This chapter treats DevOps as a socio-technical capability rather than merely a collection of automation products. It considers version control, branching, build systems, containers, infrastructure as code, continuous integration and delivery, cloud and edge deployment, observability, service reliability, rollback, and disaster recovery.

Learning Objectives

After studying this chapter, the reader should be able to:

1. Explain DevOps principles and relate them to software delivery performance.

2. Select suitable version-control and branching practices.

3. Design reproducible builds, container images, and CI/CD pipelines.

4. Compare deployment models for cloud, edge, and content delivery platforms.

5. Apply infrastructure-as-code principles and manage configuration safely.

6. Design logging, metrics, and distributed tracing for observable systems.

7. Define service-level indicators, objectives, error budgets, and reliability targets.

8. Select rollback, backup, disaster-recovery, and deployment strategies.

9. Evaluate governance, security, cost, and ethical implications of automated delivery.

1. DevOps as a Delivery and Operating Model

Traditional organisations often divide responsibility among development, testing, operations, security, and compliance teams. Handoffs create queues, delayed feedback, unclear ownership, and incentives to optimise local tasks rather than the complete service. DevOps reduces these barriers through shared responsibility, automation, small changes, rapid feedback, and operational learning.

DevOps does not mean that every engineer performs every task. It means that teams collectively own the lifecycle of a service, including its operability and consequences in production. Core principles include:

• Flow: reduce the time and friction required to move a validated change into use.
• Feedback: obtain rapid information from tests, users, security controls, and production telemetry.
• Continuous learning: use incidents and experiments to improve both technology and process.
• Automation: automate repeatable, deterministic activities while retaining human judgement for exceptional decisions.
• Shared ownership: treat reliability, security, cost, and compliance as engineering concerns.

Useful delivery measures include deployment frequency, lead time for changes, change-failure rate, and time to restore service. These measures should be interpreted together. For example, high deployment frequency with frequent severe incidents is not effective DevOps. Metrics can also create harmful incentives if used to rank individuals rather than improve systems.

A DevOps operating model normally includes trunk-based or short-lived-branch development, automated tests, immutable build artefacts, deployment automation, infrastructure as code, centralised telemetry, and a blameless approach to incident learning. Blameless does not mean responsibility-free: organisations still need accountability, control, and remediation.

2. Version Control and Change Management

Version control records the history of source code, configuration, documentation, infrastructure definitions, and sometimes database migration scripts. Distributed systems such as Git allow developers to work locally and synchronise with shared repositories. A commit should represent a coherent, reviewable change. Good commits have a clear message, limited scope, and tests that explain their intended behaviour.

A repository strategy should specify ownership, review rules, protected branches, tagging, release conventions, and treatment of secrets. Secrets must not be committed to repositories, including private repositories, because history may preserve them indefinitely. Secret-management services, short-lived credentials, and automated secret scanning are preferable.

Branching models

In trunk-based development, developers integrate small changes frequently into a main branch. Incomplete features are hidden behind feature flags rather than long-lived branches. This reduces merge divergence and supports continuous delivery, but requires strong automated tests and disciplined backward compatibility.

A feature-branch model isolates work until review and testing are complete. It may be appropriate for teams with slower review cycles, but long-lived branches increase integration risk. A release-branch model maintains a stabilisation branch while development continues. It can support formal release governance, although it creates merge and patch-management obligations.

Branching is not a substitute for release control. A mature workflow separates:

1. source integration;

2. build and verification;

3. release approval where required; and

4. production deployment.

Pull requests provide review evidence, but approval should assess architecture, security, tests, operational impact, and migration safety—not merely code style.

3. Builds, Dependencies, and Reproducibility

A build transforms source code and dependencies into a deployable artefact. A reliable build is deterministic: given the same source, declared dependencies, toolchain, and build inputs, it should produce functionally equivalent output. Reproducibility is difficult when dependencies are unpinned, timestamps are embedded, external resources are downloaded dynamically, or compiler versions vary.

Professional build systems should:

• declare dependencies explicitly;
• use lockfiles or controlled dependency versions;
• separate compilation, testing, packaging, and publishing;
• cache dependencies without allowing stale or untrusted artefacts;
• generate software bills of materials where required;
• scan dependencies and artefacts for known vulnerabilities;
• preserve provenance, including commit, builder, tool versions, and source location.

A build pipeline commonly contains formatting and static analysis, unit tests, dependency checks, integration tests, packaging, image construction, security scanning, and publication. Tests should be layered. Unit tests are fast and numerous; integration tests validate boundaries; end-to-end tests validate user journeys but are slower and more fragile. A pipeline that relies only on end-to-end tests gives feedback too late.

The build output should be promoted between environments rather than rebuilt separately for each environment. Rebuilding can produce a different artefact from the one tested. Environment-specific behaviour should be supplied through controlled configuration, not by altering the application binary.

4. Containers and Image-Based Delivery

A container packages an application with user-space dependencies while sharing the host operating system kernel. It is lighter than a virtual machine, but it is not a complete security boundary in every context. Containers improve portability and deployment consistency; they do not automatically make applications secure, stateless, or observable.

A container image consists of layers and metadata. Images should use minimal trusted base images, pin important inputs, run as a non-root user, exclude development tools and secrets, and be scanned before release. Multi-stage builds keep compilers and test utilities out of the runtime image. Image tags such as `latest` are unsuitable as release identifiers because they are mutable; immutable digests or controlled version tags provide stronger traceability.

A container orchestrator may schedule instances, perform health checks, expose services, manage secrets, and scale workloads. Readiness indicates whether an instance can receive traffic; liveness indicates whether it should be restarted. Confusing these probes can cause restart storms or route traffic to an unready process.

Containers also introduce operational responsibilities: resource requests and limits, graceful shutdown, signal handling, log output, persistent storage, network policy, image lifecycle management, and vulnerability response. A process should handle termination signals and stop accepting new work before exiting, allowing in-flight requests to complete where feasible.

5. Infrastructure as Code

Infrastructure as code (IaC) represents networks, compute resources, access policies, databases, and platform services in machine-readable definitions. It makes infrastructure reviewable, repeatable, and testable. Declarative IaC describes the desired state; the tool determines actions required to approach that state. Imperative automation instead specifies a sequence of commands. Declarative approaches generally improve convergence, but external changes and provider limitations still require careful management.

A robust IaC design separates reusable modules from environment-specific values. It maintains state securely, locks concurrent operations, protects sensitive outputs, and records changes through version control. State is operationally significant: losing or corrupting it can cause a tool to misunderstand resources and propose destructive actions.

The normal IaC workflow is plan, review, apply, and verify. Plans should be generated in CI and reviewed before production changes. Policy-as-code can reject public storage, excessive permissions, unencrypted databases, or unapproved regions. Drift detection compares actual resources with declared intent. Drift may arise from emergency changes, provider behaviour, or manual administration; it should be investigated rather than blindly overwritten.

IaC does not eliminate risk. A syntactically valid definition can still expose data, create excessive cost, or destroy a production database. Permissions, approval gates, backups, and deletion protection remain necessary.

6. Continuous Integration and Continuous Delivery

Continuous integration (CI) means that changes are merged frequently and verified automatically. Continuous delivery means that validated software is maintained in a releasable condition; deployment may require a business or operational decision. Continuous deployment automatically releases qualifying changes to production.

A production-quality pipeline commonly follows this sequence:

1. obtain the exact source revision;

2. validate formatting, dependencies, and security policy;

3. run unit and component tests;

4. build and sign an immutable artefact;

5. run integration and acceptance tests;

6. publish an artefact and software bill of materials;

7. deploy to an isolated environment;

8. run smoke and synthetic tests;

9. release progressively using an approved strategy;

10. observe health indicators and stop or reverse the change when thresholds are exceeded.

Pipeline credentials should be short-lived and scoped to the minimum required permissions. The pipeline itself is production infrastructure and must be protected against code injection, dependency substitution, untrusted pull requests, and compromised runners.

Deployment strategies

A recreate deployment stops the old version before starting the new one. It is simple but causes downtime. A rolling deployment replaces instances gradually and conserves infrastructure, although mixed versions must be compatible. A blue-green deployment maintains two environments and switches traffic between them, enabling rapid reversal at the cost of duplicated capacity. A canary deployment sends a small proportion of traffic to the new version and expands exposure after evaluation. A feature flag changes behaviour without necessarily changing the deployed binary; flags require ownership, testing, access control, and removal plans.

Database changes deserve special attention. A safe migration is often expand-and-contract: add a compatible schema element, deploy code that can use both old and new forms, migrate data, switch reads and writes, and remove obsolete elements later. A deployment rollback cannot necessarily undo a destructive database migration.

7. Cloud, Edge, and Content Delivery Deployment

Cloud deployment models include virtual machines, managed container platforms, serverless functions, managed databases, and platform-as-a-service offerings. The more infrastructure a provider manages, the less operational work the customer performs, but the customer also accepts provider constraints and dependency. The shared-responsibility model assigns physical and foundational security to the provider while leaving identity, data, application configuration, and correct use of services to the customer, with exact boundaries varying by service.

A cloud architecture should consider availability zones, regional failure, network segmentation, identity, encryption, backup, observability, cost, and portability. Multi-region deployment may reduce regional risk but increases consistency, data-transfer, operational, and testing complexity. It is not automatically superior to a well-designed single-region system.

A content delivery network (CDN) caches static assets and sometimes dynamic responses near users. Cache-control headers, validation tokens, invalidation mechanisms, compression, and versioned asset names determine correctness and efficiency. Personalised or sensitive content requires careful cache-key and header design. A CDN can also provide TLS termination, web application filtering, and edge functions.

Edge computing places computation close to users or devices to reduce latency, bandwidth use, or dependency on a central region. Edge environments may have limited resources, intermittent connectivity, heterogeneous runtimes, and difficult debugging. State should be partitioned deliberately, and security updates must reach distributed nodes. Edge deployment is appropriate only when its latency, availability, or data-locality benefits justify the additional complexity.

8. Observability: Logs, Metrics, and Traces

Monitoring asks whether a known condition is occurring. Observability asks whether internal system states can be inferred from external outputs. The principal telemetry types are logs, metrics, and traces.

Logs are timestamped records of events. Structured logs, usually encoded as key-value data, support search and aggregation. A useful event includes a timestamp, severity, service, environment, operation, correlation identifier, outcome, and safe contextual fields. Logs should avoid passwords, tokens, unnecessary personal data, and full sensitive payloads. Sampling and retention policies control volume and cost.

Metrics are numerical time series. Counters increase monotonically, gauges represent current values, and histograms represent distributions such as request latency. Histograms are generally more useful than averages for latency because an average can conceal a poor tail. High-cardinality labels, such as unrestricted user identifiers, can make a metrics system expensive and slow.

Distributed tracing follows a request across services using trace and span identifiers. A trace records the overall operation; spans represent work by individual services or dependencies. Context propagation must cross HTTP, messaging, and asynchronous boundaries. Sampling may be probabilistic, tail-based, or rule-based. Trace data can reveal queueing, retries, fan-out, and slow dependencies that service-level metrics alone cannot explain.

Telemetry must be correlated. A request identifier in logs, a trace identifier in logs and spans, and deployment metadata in metrics allow an engineer to connect a user-visible failure with a code and infrastructure change. OpenTelemetry provides widely adopted APIs and protocols, but instrumentation still requires meaningful naming, privacy controls, and operational ownership.

9. Reliability Targets and Error Budgets

A service-level indicator (SLI) is a measured representation of service performance, such as the proportion of valid requests served successfully within a latency threshold. A service-level objective (SLO) is the target value for an SLI over a stated period. A service-level agreement (SLA) is a contractual commitment, often with remedies, and is not synonymous with an internal SLO.

For an availability-style SLI, if valid requests are the denominator and successful requests are those meeting the defined success condition, then:

`Availability SLI = successful valid requests / total valid requests`

The definition must specify exclusions, time window, endpoint scope, and treatment of dependency failures. A target such as 99.9% over a 30-day month permits approximately 43.2 minutes of unavailability in a 30-day period, assuming 30 days and continuous measurement: 30 × 24 × 60 × (1 − 0.999) = 43.2 minutes. This is a planning interpretation, not a guarantee of evenly distributed downtime.

An error budget is the permitted unreliability: `error budget = 1 − SLO`, applied to the same measurement basis. When a service consumes its budget rapidly, the organisation may pause risky releases, prioritise reliability work, or reduce change scope. Error budgets create a principled connection between delivery speed and reliability, but they should not be applied mechanically when failures harm safety, privacy, or legal compliance.

Reliability engineering also considers saturation, latency, traffic, and errors. Alerting should focus on symptoms and user impact, not every internal event. A good alert is actionable, urgent enough to wake an engineer when necessary, and linked to a runbook.

10. Rollback, Incident Response, and Disaster Recovery

Rollback returns traffic or application behaviour to a known-good version. It is effective when artefacts are immutable, releases are traceable, and backward compatibility is maintained. Alternatives include roll-forward fixes, feature-flag deactivation, traffic shifting, and restoring a previous infrastructure state. Rollback should be tested; an untested rollback plan is an assumption.

Incident response includes detection, triage, containment, communication, mitigation, recovery, and learning. Incident roles may include an incident commander, operations lead, communications lead, and subject-matter experts. Status updates should be factual and avoid speculation. Post-incident reviews should identify contributing conditions and systemic improvements rather than search for an individual to blame.

Disaster recovery addresses events that exceed ordinary failure handling. Important terms are:

• Recovery point objective (RPO): the maximum tolerable amount of data loss measured in time.
• Recovery time objective (RTO): the target time to restore an acceptable service level.
• Backup: a retained copy of data or configuration.
• Replication: maintaining copies, potentially synchronously or asynchronously, in another location.

Backups must be encrypted, access-controlled, monitored, retained according to policy, and tested through restoration exercises. Replication is not a replacement for backups because corruption or deletion may be replicated. Recovery procedures should cover identity, DNS, certificates, infrastructure state, application artefacts, data, dependencies, and communication. Disaster recovery plans must be exercised under realistic assumptions, including loss of a region or provider service.

11. Security, Governance, Cost, and Ethics

DevSecOps integrates security throughout delivery. Relevant controls include least-privilege identity, protected branches, dependency and image scanning, secret management, signed artefacts, network policy, runtime detection, vulnerability remediation, and audit trails. Security gates should be risk-based. A pipeline that blocks every low-confidence finding may encourage unsafe bypasses; a pipeline that ignores critical findings creates obvious exposure.

Governance includes change records, separation of duties where required, retention policies, regulatory evidence, data residency, accessibility, and software supply-chain assurance. Automated decisions should be explainable enough for affected stakeholders and auditable after the fact. Personal data in logs and traces creates privacy risk under data-protection laws and organisational policy. Data minimisation, masking, retention limits, access reviews, and deletion procedures are therefore operational requirements.

Cloud cost is an engineering quality attribute. Idle resources, excessive telemetry, cross-region transfer, overprovisioned databases, and unbounded autoscaling can make a technically successful service financially unsustainable. Cost allocation tags, budgets, anomaly detection, capacity policies, and architecture reviews support FinOps practices.

Artificial intelligence can assist with test generation, anomaly detection, incident summarisation, and configuration analysis. However, generated code and operational recommendations require review. AI systems may reproduce insecure patterns, expose confidential telemetry to external services, or produce plausible but incorrect incident explanations. Human approval, provenance, access controls, and evaluation against historical incidents are essential.

12. Illustrative Case: Deploying a University Learning Platform

Background

A fictional university operates a learning platform used by students during normal teaching and heavily during assessment deadlines. The platform consists of a web frontend, an API, a relational database, background workers, and object storage for submissions.

Problem

Deployments occur monthly through manual server procedures. Releases require downtime, rollback takes several hours, logs are stored locally, and the operations team cannot distinguish application latency from database contention. A failed deployment during an examination period would have serious academic and reputational consequences.

Analysis

The team identifies four risk categories: inconsistent environments, untested deployment procedures, inadequate telemetry, and insufficient recovery evidence. It defines an availability SLO for submission requests, a latency SLI for interactive requests, an RPO of 15 minutes for submission metadata, and an RTO of one hour for the service. These values are illustrative objectives requiring institutional approval.

Approach

The source repository adopts protected trunk-based development, mandatory review, automated tests, and dependency scanning. The build produces a signed container image identified by commit and digest. Infrastructure definitions create separate development, staging, and production environments. Database migrations follow expand-and-contract principles.

CI builds once and promotes the same image. A staging deployment runs integration tests and synthetic submission journeys. Production uses a canary release: 5% of traffic is directed to the new API version. Automated evaluation checks error rate, p95 latency, queue depth, database connections, and submission success. If thresholds are exceeded, traffic returns to the previous version and the incident process begins.

The platform emits structured logs with trace identifiers, RED-style service metrics—rate, errors, and duration—and traces across the API, database, and worker queue. Sensitive submission content is excluded from telemetry. Database backups are encrypted and restored monthly in a separate environment. A documented regional recovery exercise validates infrastructure recreation, DNS changes, identity access, and data restoration.

Outcome

The organisation gains smaller, more frequent releases, measurable deployment evidence, faster fault isolation, and a tested recovery path. The team also discovers that its original SLO omitted background processing; it revises the service model rather than treating the first definition as final.

Learning Points

The case demonstrates that automation alone is insufficient. Reliable delivery depends on compatible data changes, observable user outcomes, controlled credentials, tested recovery, and agreement between technical and institutional stakeholders. SLOs and deployment gates should be derived from user and business consequences, not copied from generic templates.

13. Best-Practice Design Checklist

A mature DevOps and cloud deployment capability should be able to answer the following questions:

• Can every production artefact be traced to source, dependencies, builder, and approval?
• Are changes small, reviewed, tested, and reversible?
• Is infrastructure declared, peer-reviewed, policy-checked, and protected from drift?
• Are secrets excluded from code, images, logs, and traces?
• Can the system report user-facing success, latency, traffic, and failure?
• Are alerts actionable and linked to current runbooks?
• Are deployment and database rollback limitations understood?
• Are backups restorable, and have RPO and RTO been tested?
• Are cloud cost, privacy, accessibility, and regulatory requirements visible in design reviews?
• Does the organisation learn from incidents without suppressing inconvenient evidence?

Contemporary Developments

Platform engineering is emerging as a way to provide internal self-service capabilities—standard pipelines, secure runtime templates, observability, and deployment workflows—without forcing every product team to become expert in every infrastructure detail. GitOps extends declarative infrastructure practices by using a repository as the desired-state interface for deployment. Policy engines increasingly evaluate security, privacy, and cost before resources are created.

Supply-chain security is becoming more prominent through provenance attestations, signed artefacts, dependency transparency, and verification at deployment time. Serverless and edge platforms continue to reduce infrastructure management but can increase vendor dependence and complicate local testing and distributed state. Observability is also expanding toward continuous profiling, eBPF-based runtime insight, and automated correlation of telemetry. These developments should be evaluated according to operational outcomes, not novelty.

Chapter Summary

DevOps integrates development, operations, security, and governance around the complete lifecycle of a service. Version control and disciplined branching make changes traceable; reproducible builds and immutable containers make releases consistent; infrastructure as code makes cloud resources reviewable and repeatable. CI/CD pipelines automate verification and controlled delivery, while rolling, blue-green, canary, and feature-flag strategies manage deployment risk.

Cloud, CDN, and edge platforms offer different trade-offs in control, latency, availability, cost, and complexity. Observability combines structured logs, metrics, and distributed traces to explain system behaviour. SLIs, SLOs, SLAs, and error budgets connect reliability engineering with release decisions. Rollback, incident response, backups, RPO, RTO, and recovery exercises provide resilience beyond normal operation. Finally, secure supply chains, privacy-aware telemetry, cost governance, accessibility, and responsible use of AI are essential to professional cloud delivery.

Key Terms

DevOps: A socio-technical approach integrating software delivery and operations.

Continuous integration: Frequent integration of changes with automated verification.

Continuous delivery: Keeping validated software ready for release.

Container: An isolated process package sharing the host kernel.

Immutable artefact: A released object that is not modified after publication.

Infrastructure as code: Machine-readable management of infrastructure resources.

Drift: Difference between declared infrastructure state and actual state.

Canary deployment: Gradual release to a limited portion of traffic.

Feature flag: Controlled runtime switch for application behaviour.

CDN: Distributed network that delivers content near users.

Observability: Ability to infer internal system state from external outputs.

Structured log: Machine-readable event record with named fields.

Distributed trace: End-to-end record of work across services.

SLI: Measured indicator of service performance.

SLO: Target value for an SLI over a specified period.

Error budget: Permitted unreliability implied by an SLO.

RPO: Maximum acceptable data loss measured in time.

RTO: Target time to restore an acceptable service level.

Infrastructure drift: Unplanned divergence of deployed resources from code.

Software bill of materials: Inventory of components in a software artefact.

Review Questions

Short Answer Questions

1. What distinguishes continuous delivery from continuous deployment?

2. Why are immutable image digests preferable to the `latest` tag?

3. Define an SLI and an SLO.

4. What is the purpose of a readiness probe?

5. Why is replication not a complete substitute for backup?

Long Answer Questions

1. Explain how version control, reproducible builds, containers, and infrastructure as code combine to improve release traceability.

2. Compare rolling, blue-green, canary, and recreate deployment strategies, including their principal risks.

3. Explain how logs, metrics, and traces complement one another in a distributed web system.

4. Discuss the relationship among SLOs, error budgets, deployment frequency, and incident management.

5. Analyse the security, privacy, governance, and cost responsibilities of a cloud-hosted application.

Analytical / Application Questions

1. A service has a 99.95% monthly availability SLO. Assuming a 30-day month, calculate the permitted downtime and explain why this figure must be interpreted with the SLO measurement definition.

2. Design a deployment and rollback plan for a web application that changes both its API and relational database schema. Identify compatibility constraints.

3. A CDN-served application shows low average latency but frequent slow user experiences. Specify the telemetry and analysis needed to investigate the problem, including suitable latency statistics and trace dimensions.

Multiple Choice Questions with Answers

Question 1

Which practice most directly supports reproducible deployments?

A. Rebuilding the application separately in every environment

B. Promoting the same immutable artefact between environments

C. Using a mutable latest image tag

D. Installing dependencies manually on production servers

Correct Answer: B. Promoting the same immutable artefact between environments

Question 2

What is the primary purpose of a readiness probe?

A. To determine whether an instance should receive traffic

B. To encrypt application logs

C. To create database backups

D. To compile source code

Correct Answer: A. To determine whether an instance should receive traffic

Question 3

Which deployment strategy initially exposes a new version to a small proportion of users?

A. Recreate

B. Blue-green

C. Canary

D. Manual deployment

Correct Answer: C. Canary

Question 4

Which telemetry type is best suited to following one request across multiple services?

A. Gauge metric

B. Distributed trace

C. Container image

D. Infrastructure plan

Correct Answer: B. Distributed trace

Question 5

An availability SLO of 99.9% implies an error budget of:

A. 0.01%

B. 0.1%

C. 1%

D. 9.9%

Correct Answer: B. 0.1%

Question 6

What does RPO measure?

A. The time required to deploy code

B. The maximum acceptable data loss measured in time

C. The percentage of successful requests

D. The number of replicas required

Correct Answer: B. The maximum acceptable data loss measured in time

Question 7

Which practice best protects production pipeline credentials?

A. Embedding permanent keys in source code

B. Sharing one administrator credential among all jobs

C. Using short-lived, least-privilege credentials

D. Printing credentials in build logs

Correct Answer: C. Using short-lived, least-privilege credentials

Question 8

Why can averages be inadequate for reporting request latency?

A. They cannot be stored as metrics

B. They can conceal poor tail performance

C. They always exaggerate errors

D. They measure only CPU utilisation

Correct Answer: B. They can conceal poor tail performance

Question 9

What is infrastructure drift?

A. A difference between declared and actual infrastructure state

B. A technique for compressing container images

C. A method of tracing requests

D. A database indexing strategy

Correct Answer: A. A difference between declared and actual infrastructure state

Question 10

Why should database migrations be designed for backward compatibility during progressive deployment?

A. Old and new application versions may run simultaneously

B. It eliminates the need for backups

C. It prevents all security vulnerabilities

D. It makes monitoring unnecessary

Correct Answer: A. Old and new application versions may run simultaneously