Backup, Resilience and Disaster Recovery

Introduction

Cloud services reduce the need for organisations to operate physical infrastructure, but they do not eliminate failure. A cloud region can become unavailable, an administrator can delete critical data, an application can corrupt a database, or ransomware can encrypt both production systems and accessible backups. Cloud providers generally secure the underlying infrastructure; the customer remains responsible for many aspects of data protection, configuration, identity management, recovery design and testing.

Backup, resilience and disaster recovery are related but distinct disciplines. Backup creates recoverable copies of data. Resilience enables a service to continue operating, or to degrade acceptably, when components fail. Disaster recovery (DR) restores technology services and data after a disruptive event. Effective protection combines all three with business priorities, security controls and regularly rehearsed procedures.

This chapter applies these ideas to cloud environments. It explains recovery objectives, business impact analysis, redundancy patterns, immutable backups, ransomware-resistant design, recovery procedures, testing and governance.

Learning Objectives

By the end of this chapter, you should be able to:

• distinguish backup, high availability, resilience, business continuity and disaster recovery;
• define and apply recovery time objective (RTO) and recovery point objective (RPO);
• perform a basic business impact analysis;
• select suitable redundancy and backup architectures;
• design controls for immutable and isolated backups;
• describe a secure recovery workflow for outages, ransomware and data loss;
• evaluate recovery tests using evidence and measurable results; and
• identify governance, legal and ethical requirements affecting recovery decisions.

1. The Recovery Problem

A recovery plan begins with the question: What must be restored, in what order, and by when? The answer is not necessarily “everything immediately.” A public website, payment service, identity platform and archival reporting system may have different business values and recovery requirements.

A useful classification is:

Redundancy is not the same as backup. Two synchronously replicated databases may both contain an accidental deletion or encrypted files. Replication improves availability but can copy corruption rapidly. A backup should provide historical recovery points, while redundancy provides operational continuity.

Failure categories

Cloud recovery planning should consider at least four categories:

1. Component failure: a virtual machine, disk, network link or software component fails.
2. Location failure: an availability zone, data centre or cloud region becomes unavailable.
3. Logical failure: data is deleted, corrupted by software, incorrectly migrated or overwritten.
4. Adversarial failure: credentials are stolen, ransomware encrypts data, or an attacker deliberately destroys recovery resources.

Each category requires different controls. Automatic failover may address a failed server but not compromised administrator credentials. A backup may restore deleted data but cannot by itself provide uninterrupted service.

2. Business Impact Analysis and Recovery Objectives

Business impact analysis

A business impact analysis (BIA) identifies critical processes, their dependencies and the consequences of disruption. It normally records:

• the process or service;
• business owner and technical owner;
• users and dependent systems;
• maximum tolerable period of disruption (MTPD);
• financial, operational, legal, safety and reputational impacts;
• required data and acceptable data loss;
• manual workarounds; and
• restoration priority.

The BIA should be evidence-based. Interviews with managers are useful, but they should be supplemented by transaction volumes, service-level commitments, regulatory requirements and dependency mapping. A technically impressive recovery design is inappropriate if it restores low-value systems before a critical identity or payment dependency.

RTO and RPO

The recovery time objective (RTO) is the target maximum elapsed time between service disruption and restoration of an acceptable service level. The recovery point objective (RPO) is the target maximum period of data change that the organisation can afford to lose, measured backward from the disruption.

If a service has an RPO of 15 minutes, the recovery design should provide a recovery point no more than approximately 15 minutes old under stated operating conditions. RPO is not identical to backup frequency: a backup every 15 minutes can still fail to meet the objective if copying, verification or restoration takes longer or if the latest backup is unusable.

Recovery time can be represented conceptually as:

Here each term is a duration: detection, decision and authorisation, environment preparation, data and service restoration, validation, and resumption of normal operations. The relationship is a planning model rather than a universal physical law. It assumes the stages are sequential; parallel activities may reduce the elapsed time, but dependencies and approval delays must still be measured.

A recovery design must be feasible relative to the MTPD:

This inequality is meaningful only when both quantities use the same unit and describe the same service level. “Recovered” should be defined—for example, read-only access may not satisfy a requirement for complete transaction processing.

Availability and downtime

Availability is commonly expressed as:

Both uptime and downtime must be measured over the same observation period and under an agreed definition of service availability. An availability target of 99.9% over a 365-day year permits approximately 8.76 hours of unavailability, since the year contains 8,760 hours and 0.1% of that period is 8.76 hours. This is a planning estimate; maintenance exclusions, measurement windows and service-level contract definitions may produce different results.

Availability targets do not directly specify RPO. A service can be highly available while losing recent data, or have excellent backups while remaining unavailable for many hours during restoration.

Worked example: selecting a recovery design

An online order service has an MTPD of four hours. The BIA determines that it can lose at most ten minutes of orders, so its RPO is ten minutes. A proposed recovery process requires 20 minutes to detect the outage, 30 minutes to approve failover, 90 minutes to provision and restore the service, and 45 minutes to validate transactions.

The estimated recovery time is:

This is 3 hours 5 minutes and therefore fits within the four-hour MTPD, assuming the assumptions are realistic. However, the design meets the RTO only if its recovery points are no more than ten minutes old and the 90-minute restoration process can reliably achieve that point. A test should verify both conditions.

3. Backup Architecture and the 3-2-1 Principle

A traditional starting point is the 3-2-1 rule:

• maintain at least three copies of important data;
• use at least two different storage media or storage systems; and
• keep at least one copy offline or geographically separated.

In cloud environments, “media” should be interpreted meaningfully. Three buckets in the same account and region may provide little protection against account compromise or regional failure. A stronger design separates copies by account, credentials, administrative boundary and location. Some organisations extend the rule to 3-2-1-1-0: one copy immutable, and zero unverified backup errors after testing.

A backup policy should specify:

• protected workloads and exclusions;
• schedule and retention periods;
• encryption and key management;
• access roles and approval requirements;
• geographical and jurisdictional placement;
• integrity verification;
• restoration priority;
• deletion and legal-hold rules; and
• monitoring and escalation.

Backup methods

Full backup copies all selected data. It simplifies restoration but consumes more storage and time.

Incremental backup copies changes since the most recent backup of any kind. It reduces backup volume but may require a chain of backups during restoration.

Differential backup copies changes since the most recent full backup. It generally restores with one full backup and one differential backup, but the differential grows until the next full backup.

Snapshot captures the state of a volume or service at a point in time. Snapshots are often fast and operationally useful, but they may remain within the same security boundary, depend on the source platform, or lack protection from privileged deletion. A snapshot should not automatically be treated as an independent disaster-proof backup.

Continuous data protection records changes frequently or continuously, potentially offering a small RPO. It requires careful management of storage cost, performance, retention and corruption propagation.

Retention and recovery points

Retention should reflect business, legal and security needs rather than storage convenience. A practical schedule might combine frequent short-term points, daily points for several weeks, and monthly points for longer-term recovery. Long retention does not help if all copies are encrypted by the same attacker or if encryption keys and restoration instructions have been lost.

4. Immutability, Isolation and Ransomware Resistance

An immutable backup cannot be altered or deleted during a defined protection period, including by ordinary administrative actions. Technologies may implement this through write-once-read-many (WORM) storage, object-lock retention, legal holds or a separate vault with restricted controls. Immutability is a control property that must be verified; merely labelling a bucket “backup” does not make it immutable.

Important design controls include:

1. Separate administrative identities: backup operators should not automatically have authority to delete protected copies.
2. MFA and strong authentication: privileged actions should require phishing-resistant or otherwise strong authentication where feasible.
3. Cross-account or cross-subscription storage: compromise of the production account should not grant unrestricted control over backups.
4. Retention lock: retention settings should prevent unauthorised shortening of the protection period.
5. Network and endpoint isolation: backup vaults should have limited connectivity and should not be broadly mounted by production hosts.
6. Independent key protection: encryption keys require access controls, rotation policy, recovery procedures and protection from the same compromise.
7. Monitoring: deletion attempts, policy changes, unusual backup volume and mass restore activity should generate alerts.
8. Clean-room restoration: recovery should occur in a newly controlled environment rather than automatically reintroducing compromised credentials or software.

Immutability involves trade-offs. A retention lock can prevent legitimate deletion, increase costs and complicate privacy requests. The organisation therefore needs documented retention schedules, legal review and an emergency governance process. Immutability also does not guarantee correctness: corrupted data can be preserved perfectly. Integrity checks and recovery testing remain essential.

5. Redundancy and Availability Patterns

Local redundancy

Multiple instances across fault domains can tolerate a server or zone failure. Load balancing, health checks and automated replacement reduce recovery time. This pattern is appropriate when the failure is limited in scope and the application is designed to operate with partial capacity.

Synchronous replication

Synchronous replicas acknowledge a write only after multiple locations have received it. This can reduce data loss during failover, but it introduces latency and may replicate corruption or deletion. It is most suitable when locations have adequate network performance and the application supports consistent failover.

Asynchronous replication

Asynchronous replication acknowledges the primary write before the replica is confirmed. It usually supports greater distance and lower latency, but the replica may lag. The lag is a direct concern for RPO and must be measured rather than assumed.

Multi-zone and multi-region designs

A multi-zone architecture addresses failures within a region. A multi-region architecture can address regional disruption but introduces complexity in data consistency, traffic routing, identity, secrets, observability, cost and jurisdiction. Active-active systems serve users from multiple locations; active-passive systems keep a standby that is promoted after failure. Warm standby usually has preconfigured capacity, whereas cold standby requires more preparation and therefore normally has a longer RTO.

Redundancy must include dependencies. A duplicated application that depends on one unavailable identity provider, DNS service, encryption key or network route is not truly resilient. Dependency mapping should identify shared failure domains and common-mode failures.

6. Disaster Recovery Lifecycle

A robust DR capability is a lifecycle rather than a document:

1. Analyse: perform the BIA, identify threats and define RTO, RPO and service levels.
2. Design: select backup, redundancy, security and recovery patterns.
3. Implement: configure infrastructure as code, backup policies, monitoring, access controls and runbooks.
4. Validate: test restoration, failover, data integrity and staff responsibilities.
5. Operate: monitor backup jobs, replication lag, storage capacity and control changes.
6. Improve: learn from tests and incidents, revise assumptions and retest.

A recovery runbook should be specific enough for an authorised responder to use under pressure. It should identify triggers, decision authorities, communication channels, dependencies, credentials or break-glass procedures, recovery sequence, validation checks, rollback conditions and evidence requirements. Sensitive secrets should not be printed in a runbook; the document should reference an appropriately protected secrets-management process.

Recovery sequence

A generic sequence is:

• declare the incident and establish command;
• contain the threat, especially if compromise is suspected;
• preserve logs and relevant evidence;
• assess the last known clean recovery point;
• secure or rebuild the recovery environment;
• restore foundational services such as identity, networking and key management;
• restore data and application tiers in dependency order;
• validate integrity, security and business transactions;
• reconnect users gradually and monitor closely; and
• document decisions, costs, data loss and follow-up actions.

For ransomware, restoring immediately from the newest backup may reintroduce malware. The team should identify the attack timeline, isolate affected credentials and systems, select a clean recovery point, scan restored systems, rotate secrets and preserve forensic evidence. Legal, regulatory, insurance and law-enforcement obligations may apply.

7. Testing and Measurement

A plan that has never been tested is an assumption. Testing should progress from low-risk activities to realistic exercises:

Tests should record planned and actual RTO, planned and actual RPO, restoration success rate, data integrity results, unresolved dependencies, staff actions and evidence. A useful measurement is the difference between the actual and target recovery time:

Here ∆T is measured in minutes or hours. A positive value indicates that recovery exceeded the target; a negative value indicates faster recovery. The metric is meaningful only when the same service level and test scope are used. A fast restoration that produces corrupt or incomplete data is not a successful recovery.

Testing must avoid unnecessary harm. Use synthetic or masked data where possible, define abort criteria, notify stakeholders, control traffic carefully and ensure that test actions cannot delete production data. After-action reviews should assign owners and due dates for corrective actions.

8. Applied Case: Illustrative Online Retailer

Background

Northstar Retail, a fictional mid-sized online retailer, operates a customer portal, order database, payment integration, warehouse interface and analytics platform in a public cloud. Its production environment uses multiple availability zones, daily snapshots and asynchronous replication to a second region.

Problem

An administrator’s credentials are stolen through a phishing attack. The attacker disables monitoring, encrypts production data and attempts to delete snapshots. Some snapshots are protected by retention lock, but the security team has not performed a full application restoration for nine months. The business estimates that more than six hours of outage would cause serious contractual and reputational damage. It can tolerate no more than 15 minutes of lost orders.

Analysis

The BIA identifies the order and payment workflow as the highest priority. The analytics platform can be restored later. The existing multi-zone design limits the effect of a single-zone failure but does not address credential compromise. Asynchronous replication may have copied encrypted data, so it cannot automatically be treated as a clean recovery source. The immutable vault is valuable, but its recovery procedures, key access and application dependencies require validation.

Approach and solution

Northstar’s incident team first isolates compromised accounts, blocks suspicious network paths and preserves logs. It declares a security incident rather than an ordinary availability event. The team selects an immutable recovery point from before the attack, verifies its integrity, and creates a clean recovery account with separately controlled administrative access. Identity, networking, secrets and database services are restored first; the order service follows, then warehouse integration and customer access.

The organisation introduces cross-account immutable backups, separate security monitoring, MFA for privileged actions, quarterly restoration tests and a recovery dashboard showing backup age, replication lag and failed jobs. It documents a 15-minute RPO as a measured objective rather than merely a schedule. Payment reconciliation is added to validation because technical availability alone cannot prove that orders and payments are consistent.

Outcome

A controlled exercise restores the order service in four hours and 20 minutes, within the six-hour MTPD, but the first test reveals an unavailable encryption key and an undocumented warehouse dependency. After remediation, a second exercise restores the service in three hours and 35 minutes. The exercise also demonstrates that the backup system can meet the data-loss objective only when its monitoring detects a failed replication job promptly.

Learning points

• Replication and snapshots did not replace immutable, isolated recovery points.
• Identity, keys and dependencies were as important as application servers.
• RTO and RPO were validated through evidence, not assumed from product features.
• Recovery from ransomware requires containment, clean restoration and credential rotation.
• Testing exposed weaknesses that design documentation had not revealed.

9. Governance, Ethics and Regulation

Recovery decisions affect privacy, safety, contractual obligations and individual rights. Organisations should consider applicable data-protection, financial, health, critical-infrastructure and records-retention requirements. Copies may cross national borders, and backup retention may conflict with data minimisation or deletion requests. Legal counsel and privacy officers should help define retention, access, litigation hold and deletion processes.

Access to backups should follow least privilege and segregation of duties. A backup administrator who can both create and destroy all copies represents a concentration of risk. Recovery logs should be protected because they may contain personal data, credentials, IP addresses or sensitive business information.

Ethically, organisations should communicate honestly about data loss and service interruption. They should not claim that a service is “fully recovered” when only limited functionality is available. Recovery priorities should be based on documented business and human impact rather than solely on the loudest stakeholder or most profitable customer.

10. Contemporary Developments and Practical Guidance

Cloud-native recovery increasingly uses infrastructure as code, policy-as-code, automated compliance checks and disposable recovery environments. These tools improve repeatability, but automation can also amplify errors. Changes to backup policies and recovery scripts require code review, testing and controlled deployment.

Artificial intelligence can assist with anomaly detection, dependency discovery, incident triage and prediction of unusual backup activity. It should not be granted unsupervised authority to delete backups, alter retention or restore systems in a security incident. AI-generated recommendations require human review, reliable data and auditable decisions.

A practical checklist is:

• identify critical services and owners;
• document dependencies and shared failure domains;
• set RTO, RPO and MTPD from a BIA;
• maintain geographically and administratively separated copies;
• make at least one copy immutable and monitor attempts to change it;
• protect keys, identities and recovery accounts separately;
• verify backups and test complete application restoration;
• maintain clear, current runbooks;
• measure actual recovery performance; and
• improve controls after every test and incident.

Chapter Summary

Backup preserves historical data, resilience limits disruption, and disaster recovery restores services after major failure. Effective planning begins with a business impact analysis that identifies critical processes, dependencies, MTPD, RTO and RPO. Availability and replication reduce interruption but do not replace independent historical backups.

Strong cloud recovery designs use separation of accounts and locations, immutable retention, least privilege, protected keys, monitoring and tested restoration. Recovery procedures must address ordinary outages, logical corruption and hostile events such as ransomware. Tests should measure actual recovery time, data loss, integrity and dependency performance. Governance must reconcile resilience with privacy, legal retention, jurisdiction, ethical communication and accountability.

Key Terms

Backup: A recoverable copy of data or system state.

Business impact analysis: A structured assessment of disruption consequences and recovery priorities.

Business continuity: The capability to continue essential business activities during disruption.

Disaster recovery: Processes and technology used to restore services after a disruptive event.

High availability: Design that reduces service interruption through redundancy and failover.

Immutable backup: A backup protected from alteration or deletion for a defined period.

Isolation: Separation of recovery resources from production and common administrative control.

MTPD: Maximum tolerable period of disruption for a process or service.

RPO: Maximum acceptable age of recoverable data after an incident.

RTO: Target maximum time to restore an acceptable service level.

Redundancy: Additional components or locations that provide alternate capacity or function.

Replication: Copying data between systems or locations.

Snapshot: A point-in-time representation of data or storage state.

Synchronous replication: Replication in which writes are confirmed at required replicas before acknowledgement.

Asynchronous replication: Replication in which the replica may lag behind the primary system.

WORM: Write-once-read-many storage that prevents modification during retention.

Clean-room recovery: Restoration in a newly controlled environment isolated from suspected compromise.

Failover: Transfer of service operation to an alternate component or location.

Review Questions

5 Short Answer Questions

1. Distinguish backup from redundancy.
2. Define RTO and RPO.
3. Why can replication copy ransomware damage?
4. State the traditional 3-2-1 backup rule.
5. Why should recovery procedures be tested?

5 Long Answer Questions

1. Explain how a BIA informs backup and DR priorities.
2. Compare synchronous and asynchronous replication, including their limitations.
3. Describe a ransomware-resistant cloud backup architecture.
4. Explain the stages of a disaster recovery lifecycle.
5. Discuss legal, privacy and ethical issues in backup retention.

3 Analytical / Application Questions

1. A service has an MTPD of five hours. Its proposed recovery takes 25 minutes to detect, 35 minutes to approve, 150 minutes to restore and 40 minutes to validate. Determine whether the proposed process fits the MTPD and identify one assumption that should be tested.
2. An organisation has three snapshots in one cloud account and region, all accessible by production administrators. Evaluate this design against isolation and ransomware requirements and propose improvements.
3. Design a recovery test for an online payment service with a 30-minute RPO and two-hour RTO. Specify scope, safety controls, validation evidence and success criteria.

Multiple Choice Questions with Answers

Question 1

Which statement best distinguishes RPO from RTO?

A. RPO measures acceptable data loss; RTO measures time to restore service.

B. RPO measures service availability; RTO measures backup retention.

C. RPO measures recovery cost; RTO measures encryption strength.

D. RPO measures geographic separation; RTO measures replication distance.

Correct Answer: A. RPO measures acceptable data loss; RTO measures time to restore service.

Question 2

A service has an MTPD of four hours. Detection takes 20 minutes, approval 30 minutes, restoration 90 minutes and validation 45 minutes. What is the estimated recovery time?

A. 145 minutes

B. 155 minutes

C. 185 minutes

D. 215 minutes

Correct Answer: C. 185 minutes

Question 3

Which control most directly prevents an administrator from deleting a protected backup during its retention period?

A. Load balancing

B. Retention lock or WORM protection

C. Synchronous database replication

D. A larger production disk

Correct Answer: B. Retention lock or WORM protection

Question 4

Why is replication alone not a substitute for backup?

A. Replication always reduces availability.

B. Replicas cannot be encrypted.

C. Replication may copy corruption or deletion.

D. Replicas cannot operate in another region.

Correct Answer: C. Replication may copy corruption or deletion.

Question 5

Which activity is the primary purpose of a business impact analysis?

A. Selecting a cloud provider’s logo and branding

B. Identifying disruption consequences and recovery priorities

C. Replacing all human decision-making during incidents

D. Calculating only monthly storage costs

Correct Answer: B. Identifying disruption consequences and recovery priorities

Question 6

Which recovery approach is most appropriate when ransomware may have compromised production?

A. Restore immediately from the newest accessible copy without investigation.

B. Reconnect all production credentials before restoring data.

C. Select a verified clean point and restore in an isolated environment.

D. Delete all logs to prevent disclosure.

Correct Answer: C. Select a verified clean point and restore in an isolated environment.

Question 7

Availability is commonly calculated as which expression?

A. Downtime ÷ uptime

B. Uptime ÷ (uptime + downtime)

C. RPO ÷ RTO

D. Backups × replicas

Correct Answer: B. Uptime ÷ (uptime + downtime)

Question 8

Which statement about asynchronous replication is correct?

A. The replica may lag behind the primary.

B. It guarantees zero data loss.

C. It cannot operate across regions.

D. It prevents logical corruption from spreading.

Correct Answer: A. The replica may lag behind the primary.

Question 9

What is the main value of a clean-room recovery environment?

A. It increases production write speed.

B. It isolates restoration from suspected compromise.

C. It eliminates the need for backups.

D. It automatically satisfies all regulatory duties.

Correct Answer: B. It isolates restoration from suspected compromise.

Question 10

Which evidence most convincingly demonstrates recovery capability?

A. A marketing statement from the cloud provider

B. A current backup-job success icon only

C. A documented and measured restoration test

D. A list of unused storage capacity

Correct Answer: C. A documented and measured restoration test