7. Post-Incident Learning and Problem Management

Introduction and Context

In Site Reliability Engineering, an incident is not merely an interruption to service. It is an opportunity to examine how a sociotechnical system behaved under pressure and how the organisation responded. A service outage may result from a code defect, but it may also reflect ambiguous ownership, inadequate observability, unsafe deployment practices, excessive operational load, or a design that made recovery difficult. Effective SRE therefore treats incident resolution as only the first phase of reliability work. The more durable value is created after service has been restored, when evidence is converted into learning and learning into controlled improvement.

Post-incident learning connects operational experience with engineering and management decisions. It includes the reconstruction of events, analysis of technical and organisational conditions, evaluation of response quality, and creation of corrective actions. Problem management extends this work by ensuring that recurring or significant underlying problems are identified, prioritised, tracked, and resolved or consciously accepted. Together, these practices prevent organisations from repeatedly paying the same operational cost.

A mature post-incident process does not seek a convenient individual to blame. It asks how reasonable people, working with the information and constraints available at the time, came to make decisions that produced an undesirable outcome. This does not eliminate accountability. Rather, it relocates accountability from individual punishment to system stewardship: leaders and teams remain responsible for improving the conditions in which work is performed.

Learning Objectives

After studying this chapter, the student should be able to:

• explain the purpose and principles of blameless post-incident reviews;
• distinguish incidents, problems, root causes, contributing factors, triggers, and latent conditions;
• construct reliable incident timelines from multiple evidence sources;
• apply Five Whys, causal analysis, and systems-theoretic approaches appropriately;
• design corrective actions that address both technical and organisational conditions;
• prioritise reliability work using risk, customer impact, effort, and learning value;
• establish problem-management workflows, knowledge practices, and measurable improvement;
• evaluate ethical, governance, and professional issues in post-incident investigation.

From Service Restoration to Organisational Learning

Incident response is concerned primarily with reducing current customer impact. Post-incident review is concerned with understanding what happened and improving future performance. These goals are related but not identical. A technically successful recovery may still reveal weak detection, confused decision rights, unsafe emergency changes, or excessive dependence on one expert. Conversely, an incident that was resolved quickly may not require an extensive investigation if its causes and controls are already well understood.

An incident is an event that causes, or has the potential to cause, degradation of service, loss of availability, impaired performance, security exposure, data integrity problems, or violation of a service objective. A problem is the underlying condition, or set of conditions, that contributes to one or more incidents. An incident is therefore an observed operational outcome, whereas a problem is an object of investigation and control. One problem may produce many incidents, and one incident may expose several problems.

Post-incident learning has three purposes. First, it improves the system by reducing the likelihood or impact of recurrence. Second, it improves response capability through better alerts, runbooks, roles, communication, and recovery mechanisms. Third, it improves organisational judgement by revealing assumptions and trade-offs that were previously invisible. The third purpose is often neglected. Organisations may repair a configuration while leaving intact the workload, incentives, and approval processes that made the configuration failure likely.

The principle of blamelessness must be understood carefully. A blameless review does not mean that all actions are acceptable or that deliberate negligence is ignored. It means that the primary investigative stance is curiosity rather than accusation. If participants fear dismissal, public humiliation, or disciplinary consequences for honest testimony, they will provide incomplete accounts and the organisation will learn less. Matters involving malicious conduct, fraud, or intentional policy violations may require separate formal processes, but the reliability investigation should still preserve factual integrity and avoid premature attribution.

A Vocabulary for Causal Thinking

Several terms are essential for disciplined analysis. A trigger is the immediate event that activates a failure path, such as a deployment, traffic increase, certificate expiry, or dependency failure. A cause is a condition that materially contributes to the outcome. A contributing factor increases the likelihood or severity of the incident but may not be sufficient by itself. A latent condition is a background weakness, such as inadequate testing, unclear ownership, or a misleading dashboard, that remains dormant until combined with other conditions.

A root cause is commonly described as a fundamental cause whose removal would prevent recurrence. The term is useful when it identifies an actionable condition, but it can be misleading if it implies that every incident has one single, discoverable cause. Complex services generally fail through interacting conditions. A more useful question is: which conditions could the organisation realistically change, and which changes would most reduce risk?

A control is a mechanism intended to prevent, detect, contain, or recover from an unwanted outcome. Controls may be technical, procedural, or organisational. Examples include automated validation, rate limits, health checks, peer review, change freezes, escalation policies, and tested backups. A control can fail because it is absent, poorly designed, incorrectly implemented, bypassed, or too difficult to use under pressure.

A near miss is an event that could have caused significant harm but did not, perhaps because a safeguard operated successfully. Near misses are valuable because they offer evidence about system vulnerability without the full cost of an outage. Mature teams record and analyse them rather than treating them as irrelevant successes.

The Post-Incident Review as a Learning Process

A post-incident review begins with preparation, not with a meeting invitation. The incident commander or assigned facilitator should define the scope, identify participants, preserve relevant evidence, and decide which questions the review must answer. Participants may include responders, service owners, developers, security or compliance specialists, customer-support representatives, and managers who can remove organisational barriers. Attendance should be based on knowledge and decision relevance rather than hierarchy.

The review should normally address four dimensions:

1. Impact: What customers, users, internal teams, or business processes experienced, and for how long?

2. Detection and response: How was the incident detected, escalated, diagnosed, mitigated, and communicated?

3. System conditions: What technical, procedural, and organisational conditions enabled or amplified it?

4. Improvement: Which changes will reduce likelihood, shorten detection and recovery time, or limit impact?

The facilitator should establish behavioural norms: assume good intent, distinguish observations from interpretations, use evidence, allow disagreement, avoid interrupting, and record uncertainties explicitly. A review is not a performance hearing. It should be psychologically safe enough for participants to disclose confusing alerts, incomplete procedures, unsuccessful experiments, and decisions that made sense at the time.

A useful review document contains an executive summary, customer impact, severity and service-level consequences, timeline, detection and response assessment, causal analysis, contributing factors, corrective actions, owners, due dates, and open questions. It should also record what went well. Effective alerts, helpful automation, clear communication, and successful rollback are controls worth preserving and generalising.

Constructing a Trustworthy Timeline

The timeline is the factual spine of the review. It should show not only system events but also human observations, decisions, actions, and communications. A technically precise event sequence can still be analytically weak if it omits when responders first saw an alert, when uncertainty was reduced, or when a risky decision was made.

Timeline construction usually combines logs, metrics, traces, deployment records, tickets, chat transcripts, incident-management tools, customer reports, and personal notes. Sources may disagree because clocks are unsynchronised, events are sampled, or people describe an observation in local time. The analyst should retain source timestamps, normalise them to a common reference, and mark approximate times where precision is not justified.

A useful format includes:

| Time | System or human event | Evidence source | Interpretation or uncertainty |

|---|---|---|---|

| 10:02 | Error rate begins to rise | Metrics | First measurable symptom |

| 10:07 | Alert page acknowledged | Incident tool | On-call became aware |

| 10:15 | Deployment rollback initiated | Deployment record | Mitigation action |

| 10:24 | Error rate returns to baseline | Metrics | Recovery indicator |

The distinction between event and interpretation is important. “Database connection pool reached configured limit” is an observation. “The database was the root cause” is an interpretation requiring further evidence. Recording uncertainty prevents the narrative from becoming artificially coherent after the fact.

The timeline should include counterfactual questions: When could the incident first have been detected? When was customer impact avoidable? Which action shortened the incident? Which action introduced additional risk? Such questions connect chronology to improvement without turning the timeline into a search for personal error.

Analytical Approaches: From Five Whys to Systems Theory

Five Whys

Five Whys is a questioning technique that moves from an observed outcome toward deeper conditions. For example:

• Why did requests fail? A service returned errors after exhausting its connection pool.
• Why was the pool exhausted? A new request pattern held connections longer than expected.
• Why was the pattern introduced? A feature bypassed an established asynchronous interface.
• Why was this not detected before release? Load testing did not model the relevant concurrency.
• Why did testing omit that condition? Performance ownership and representative workload definitions were unclear.

The value is not the number five. The value lies in moving beyond the immediate technical symptom. Five Whys becomes weak when it produces a single linear story, stops at “human error,” or asks why only after the fact without considering the information available to the people involved. A good analysis may produce several branches rather than one chain.

Causal and Contributing-Factor Analysis

Causal analysis maps relationships among triggers, conditions, controls, and outcomes. A fault tree begins with an undesirable top event and decomposes the combinations that could produce it. A fishbone or cause-and-effect diagram groups possible contributors under categories such as technology, process, people, environment, measurement, and external dependency. These methods are useful for broad exploration, but the resulting hypotheses must be supported by evidence.

A practical causal map may distinguish four layers:

• Outcome: customer-visible degradation or loss.
• Proximate mechanisms: failed component, saturation, incorrect routing, or invalid data.
• Enabling conditions: weak validation, missing capacity margin, unsafe defaults, or inadequate permissions.
• Organisational conditions: delivery pressure, unclear ownership, fragmented tooling, or insufficient investment.

This layered view avoids confusing the last visible failure with the most useful improvement target. It also helps identify multiple controls. If a deployment caused a failure, prevention may involve testing and progressive delivery; detection may involve a more specific alert; containment may involve automated rollback; and recovery may involve a simpler restoration procedure.

Systems-Theoretic Analysis

Systems-theoretic approaches, including Systems-Theoretic Accident Model and Processes (STAMP), treat accidents as failures of control rather than merely failures of components. A service is viewed as a hierarchy of controllers and controlled processes. Engineers, incident managers, deployment systems, policies, and monitoring tools all participate in control. Accidents can occur when control actions are missing, unsafe, delayed, based on an inaccurate model, or applied within inadequate constraints.

This approach is particularly valuable for distributed systems and organisations where no single component explains the outcome. For example, an automated deployment controller may follow its policy correctly while the policy itself permits an unsafe change. A manager may approve a release because the dashboard shows healthy indicators, while the dashboard omits a critical customer segment. The analysis therefore asks whether control algorithms, feedback, constraints, and models were adequate.

Systems-theoretic analysis is more demanding than a simple causal chain. It requires attention to interactions, feedback loops, adaptation, and conflicting goals. It is appropriate for high-severity incidents, repeated failures, complex socio-technical systems, and events involving automation or organisational coordination. It may be excessive for a small, well-understood defect with an obvious and proportionate fix.

Choosing the Method

No method is universally superior. Five Whys is accessible and efficient for a focused issue. Causal mapping is useful when several contributing factors interact. Fault trees support structured analysis of combinations and barriers. Systems-theoretic analysis is appropriate when control relationships, policies, automation, or organisational boundaries are central. Teams may combine methods, provided that the analysis remains evidence-based and does not become a decorative collection of diagrams.

Turning Findings into Corrective Action

The principal output of a review is not the document; it is changed operational capability. Corrective actions should be specific, testable, assigned to an accountable owner, and connected to a risk or learning statement. “Improve monitoring” is weak because it does not define what will change. “Create an alert for sustained queue growth, validate it against a replayed workload, document the response, and review its false-positive rate after thirty days” is stronger.

Actions commonly fall into four categories:

• Prevention: reduce the probability of recurrence through design changes, validation, capacity planning, or safer defaults.
• Detection: identify abnormal conditions earlier through instrumentation, synthetic checks, or improved alert logic.
• Containment: limit blast radius using isolation, rate limiting, feature flags, progressive delivery, or circuit breakers.
• Recovery and learning: shorten restoration through tested backups, rollback automation, runbooks, training, and knowledge capture.

An action should also specify its completion evidence. Evidence might be a merged code change, a tested runbook, a dashboard reviewed by a second team, a game-day result, or a measured reduction in alert-to-diagnosis time. Closure based only on a ticket status encourages superficial compliance.

Prioritising the Reliability Backlog

Post-incident actions compete with product development, security work, technical debt, and routine operations. Prioritisation should therefore be explicit. A practical model considers expected risk reduction, customer impact, recurrence likelihood, blast radius, time to benefit, implementation effort, dependencies, and regulatory significance. Quantitative scoring can support comparison, but it should not create false precision.

One simple prioritisation expression is:

Priority score = (impact × likelihood × exposure × control weakness) ÷ effort

The factors may be rated on an agreed scale. The formula is not a substitute for judgement; it is a way to make assumptions visible. High-risk items should be escalated when they exceed the team’s capacity. Some actions should be treated as reliability commitments tied to service-level objectives or error budgets. Others may be accepted temporarily, provided that the decision, owner, expiry date, and compensating controls are documented.

A reliability backlog should retain the relationship between the action and the incident or risk that generated it. It should contain status, owner, target date, dependencies, evidence of completion, and validation results. Leaders should review ageing and repeatedly deferred items. Repeated deferral is itself evidence that the organisation’s priorities, capacity, or incentives require attention.

Problem Management in Practice

Problem management is the disciplined lifecycle through which recurring or significant causes are identified and controlled. It begins with problem identification from incidents, trends, near misses, monitoring data, audit findings, or engineering concerns. The problem is then recorded with a clear statement, known symptoms, affected services, related incidents, and current workarounds.

Investigation may proceed through a known-error stage when the cause or workaround is sufficiently understood but permanent resolution is pending. A workaround reduces impact without removing the problem. A permanent fix changes the underlying condition and should be validated under realistic conditions. After implementation, the problem record should be reviewed for recurrence and formally closed only when evidence supports closure.

Problem management should not become a bureaucratic duplication of incident management. Its value lies in pattern recognition and cross-incident coordination. A service team may see isolated memory alerts, while a problem manager identifies a common library defect across several services. Likewise, separate incidents may reveal a systemic weakness in deployment controls. Problem management provides the organisational view needed to detect such patterns.

Useful indicators include the number of repeat incidents, mean time from incident to problem identification, age of open problems, percentage of actions completed on time, recurrence after closure, and the proportion of high-severity incidents with validated corrective actions. These metrics must be interpreted carefully. A rising number of recorded problems may indicate better detection rather than worse reliability.

Knowledge Management and Measurable Improvement

Learning becomes organisational only when it is accessible and reusable. Post-incident documents should be searchable, concise enough to consult, and linked to runbooks, service ownership records, architecture diagrams, dashboards, and change procedures. Knowledge should be written for future responders who lack the context of the original incident.

A useful knowledge article states the symptom, scope, diagnostic signals, safe immediate actions, escalation path, known limitations, and links to deeper technical material. Sensitive information should be handled according to security and privacy requirements. Knowledge should be reviewed periodically because systems, ownership, and dependencies change.

Measurement should cover both outcomes and capabilities. Outcome measures include service availability, error-budget consumption, recurrence rate, customer-impact duration, and data-loss exposure. Capability measures include alert quality, time to acknowledge, time to identify, time to mitigate, rollback success rate, runbook usage, and exercise performance. A single metric can be misleading: a low mean time to recovery may result from frequent manual interventions that create future risk.

Improvement should follow a feedback cycle: observe, analyse, change, validate, and standardise. Validation may involve chaos experiments, disaster-recovery exercises, load tests, deployment simulations, or review of subsequent incidents. The objective is not merely to complete actions but to demonstrate that the system is more resilient.

Illustrative Case: The Checkout Availability Incident

Background

The fictional company Northstar Retail operates a cloud-based checkout platform. Its SRE team maintains an error budget and uses progressive deployment for major services. The checkout service depends on a payment-authorisation service and a shared connection pool.

Situation

During a promotional campaign, checkout errors rose from less than one percent to approximately twelve percent. Customers could browse products but could not complete purchases. The on-call engineer received a generic latency alert, while customer support reported payment failures several minutes later. A rollback reduced errors, and service was restored after thirty-two minutes. No payment data was lost, but abandoned transactions increased substantially.

Analysis

The timeline showed that a new checkout feature increased the number of sequential calls to the payment service. The feature had passed unit and ordinary integration tests. Load testing used average traffic and did not reproduce the campaign’s concurrency or the payment provider’s slower response time. The connection pool reached its limit, causing requests to queue and time out.

The immediate trigger was the production deployment. Contributing factors included an unrepresentative performance test, a pool configuration that lacked an adaptive safeguard, an alert based on aggregate latency rather than queue saturation, and the absence of an automated rollback threshold. Organisational analysis showed that ownership of performance-test workloads was unclear and that the release review focused on functional correctness because the team was approaching a marketing deadline.

A Five Whys analysis that stopped at “the engineer released inefficient code” would have been inaccurate and unhelpful. The engineer followed the existing review and testing process. A broader causal analysis identified weaknesses in prevention, detection, containment, and recovery.

Approach and Solution

The team introduced a concurrency-based performance test using production-like traffic patterns, added a connection-pool saturation alert, and configured progressive rollout with automatic pause when checkout error rates exceeded a defined threshold. The service was changed to use a bounded asynchronous interaction with the payment provider. A runbook was updated to distinguish payment-provider latency from local pool exhaustion. Ownership for representative workloads was assigned jointly to the checkout team and the performance-engineering group.

Actions were prioritised according to customer impact and recurrence risk. The automated rollout guard and saturation alert were implemented first because they reduced blast radius and detection delay. The workload-model improvement followed, because it required coordination but addressed a deeper prevention weakness. A game-day exercise then tested rollback, alert interpretation, and communication with customer support.

Outcome

In a subsequent controlled load exercise, the new alert detected saturation before customer-visible failure, and the progressive deployment halted at a small traffic percentage. The organisation did not claim that the problem could never recur. Instead, it demonstrated improved prevention, earlier detection, and faster containment. The incident record remained linked to the reliability backlog and to the new performance-testing knowledge article.

Learning Points

The case illustrates that the deployment was a trigger, not a complete explanation. It also shows why corrective action should address multiple control layers. Better tests reduce likelihood, better alerts improve detection, progressive delivery limits impact, and a tested runbook improves recovery. Finally, assigning ownership and validating changes are as important as writing the review.

Challenges, Risks, and Common Mistakes

One common mistake is hindsight bias: after the outcome is known, a decision appears obviously wrong. Reviewers should ask what information, tools, and constraints were available at the time. Another mistake is the “root-cause fallacy,” in which a single technical defect is treated as sufficient explanation. Complex failures usually require attention to interacting conditions.

A third mistake is producing too many low-value actions. A long list creates the appearance of diligence while overwhelming the team. Actions should be limited, prioritised, and tied to evidence. The opposite mistake is closing actions without validation. A completed ticket is not proof of reduced risk.

Blamelessness can also be misunderstood as avoidance of responsibility. Leaders must still address unsafe systems, deficient training, repeated disregard of controls, or conflicts between delivery pressure and reliability. The distinction is between learning-oriented investigation and punitive attribution. Another risk is excessive confidentiality. If lessons are hidden from teams that could benefit, the same problem may recur elsewhere.

Technical risks include incomplete logs, missing trace context, clock skew, mutable dashboards, and retention policies that delete evidence too early. Organisational risks include unclear service ownership, fragmented teams, incentive structures that reward delivery while ignoring operability, and chronic on-call fatigue. These conditions should be recorded as reliability risks rather than treated as personal shortcomings.

Governance, Ethics, and Professional Responsibility

Post-incident records may contain customer information, credentials, security details, or employee communications. Access, retention, redaction, and distribution must follow organisational security and privacy policies. Security incidents may require legal notification, preservation of evidence, and restricted investigative channels. Reliability learning should not compromise forensic integrity.

Professional responsibility requires accuracy, fairness, and proportionality. Analysts should distinguish fact from hypothesis, acknowledge uncertainty, avoid selective quotation, and invite affected parties to review interpretations. Corrective actions should not transfer unreasonable risk to customers, junior staff, or other teams. If a known serious risk is accepted, the decision should be explicit and made at an appropriate governance level.

A review should also consider accessibility and inclusion. Communication practices, time zones, language, disability accommodations, and psychological safety affect who can participate and whose evidence is heard. A process that consistently privileges the loudest or most senior participant may reproduce blind spots.

Automation, Analytics, and Emerging Practice

Modern incident analysis increasingly uses centralised event stores, distributed tracing, automated timeline generation, change correlation, and anomaly detection. Machine-learning tools can help cluster related incidents, identify unusual dependency behaviour, or suggest likely causal relationships. These tools are aids to investigation, not authoritative judges. Models may reproduce incomplete instrumentation, confuse correlation with causation, or expose sensitive information.

Event-driven problem management can automatically create candidate problem records when repeated symptoms cross a threshold. Reliability platforms can link alerts to services, ownership, recent changes, runbooks, and historical incidents. Natural-language tools can summarise chat transcripts, but a human must verify chronology, remove speculation, and protect confidential data.

Emerging practice also emphasises resilience engineering and learning from everyday work, not only from failure. Teams study how operators adapt successfully under variable conditions and use that knowledge to improve system design. This complements, rather than replaces, incident investigation. As autonomous remediation becomes more common, reviews must examine automation decision boundaries, override mechanisms, model drift, and the possibility that automated recovery may conceal worsening conditions.

Best Practices for SRE Teams

A strong post-incident and problem-management capability generally follows these practices:

• make review scope proportionate to impact, novelty, and recurrence risk;
• preserve evidence and separate observed facts from interpretations;
• use a skilled facilitator who can manage power differences and disagreement;
• examine technical, human, procedural, and organisational conditions together;
• identify what worked as well as what failed;
• create a small number of specific, owned, measurable actions;
• prioritise actions through explicit risk and customer-impact criteria;
• validate completed changes through tests, exercises, or operational evidence;
• link incidents, problems, known errors, runbooks, and architecture records;
• monitor recurrence, action ageing, and capability improvement;
• share lessons widely while protecting security, privacy, and due process.

Chapter Summary

Post-incident learning transforms operational disruption into improved reliability. A blameless review creates the conditions for honest evidence gathering while preserving professional accountability. Reliable analysis depends on a carefully constructed timeline, clear distinctions among triggers, causes, contributing factors, controls, and latent conditions, and the appropriate use of Five Whys, causal mapping, fault analysis, and systems-theoretic reasoning.

Problem management extends individual incident reviews by identifying recurring or systemic conditions, maintaining known errors and workarounds, and coordinating permanent improvement. Corrective actions should address prevention, detection, containment, and recovery; be prioritised through risk and effort; and be validated rather than merely marked complete. Knowledge management makes learning reusable, while metrics reveal whether reliability capability and customer outcomes are improving. Ethical governance, privacy, security, psychological safety, and responsible use of automation are integral to the process.

Key Terms

Blameless review: A learning-oriented investigation that avoids premature personal blame while maintaining accountability for system improvement.

Incident: An event that causes or threatens service degradation, loss, or unacceptable risk.

Problem: An underlying condition that contributes to one or more incidents.

Trigger: An immediate event that activates a failure path.

Contributing factor: A condition that increases the likelihood or severity of an incident.

Latent condition: A background weakness that remains dormant until combined with other factors.

Root cause: A fundamental, actionable condition whose removal can reduce recurrence; not necessarily a single cause.

Control: A technical, procedural, or organisational mechanism for prevention, detection, containment, or recovery.

Near miss: An event that could have caused significant harm but did not.

Timeline: An evidence-based sequence of system events, human observations, decisions, and actions.

Five Whys: An iterative questioning method used to move from symptoms toward deeper conditions.

Causal map: A representation of relationships among triggers, conditions, controls, and outcomes.

STAMP: A systems-theoretic approach that analyses accidents as failures of control within a sociotechnical system.

Corrective action: A specific change intended to reduce risk or improve response capability.

Reliability backlog: A prioritised set of reliability improvements awaiting implementation.

Known error: A documented problem whose cause or workaround is sufficiently understood while permanent resolution is pending.

Workaround: A temporary measure that reduces impact without removing the underlying problem.

Error budget: The allowable unreliability associated with a service-level objective.

Psychological safety: A condition in which people can report information, uncertainty, and mistakes without fear of inappropriate punishment.

Knowledge management: The creation, organisation, maintenance, and reuse of operational learning.

Review Questions

Short Answer Questions

1. Distinguish an incident from a problem in SRE.

2. Why should a timeline separate observations from interpretations?

3. What is the purpose of blamelessness in a post-incident review?

4. Name four categories of corrective action.

5. What is a known error, and how does it differ from a workaround?

Long Answer Questions

1. Explain how Five Whys can be used effectively and identify its principal limitations.

2. Describe a complete post-incident review process from preparation through action validation.

3. Compare causal mapping with systems-theoretic analysis in the investigation of complex failures.

4. Explain how problem management complements incident management in a large organisation.

5. Discuss how reliability backlogs should be prioritised and governed.

Analytical / Application Questions

1. A deployment causes elevated latency only during a regional traffic surge. The deployment passed functional tests, monitoring shows aggregate health, and rollback restores service. Construct a layered causal analysis and propose actions for prevention, detection, containment, and recovery.

2. A team has completed twenty corrective-action tickets, but the same class of incident continues to recur. Analyse possible weaknesses in its post-incident process and propose measures to validate actual improvement.

3. An automated remediation system repeatedly restarts a service, reducing visible downtime but causing data-processing delays. Design a post-incident investigation using systems-theoretic concepts, including control constraints, feedback, and human oversight.

22. Multiple Choice Questions with Answers

Question 1

Which statement best distinguishes a problem from an incident?

A. An incident is always caused by a problem, whereas a problem is always caused by a person.

B. An incident is an observed operational event, while a problem is an underlying condition contributing to one or more incidents.

C. An incident is recorded after resolution, while a problem is recorded before detection.

D. An incident concerns technology only, while a problem concerns management only.

Correct Answer: B. An incident is an observed operational event, while a problem is an underlying condition contributing to one or more incidents.

Question 2

Why should a post-incident timeline distinguish an observed event from an interpretation?

A. To make the review longer and more formal

B. To ensure that all interpretations are treated as facts

C. To prevent unsupported causal conclusions from being mistaken for evidence

D. To exclude human decisions from the timeline

Correct Answer: C. To prevent unsupported causal conclusions from being mistaken for evidence

Question 3

Which is the strongest example of a corrective action?

A. Improve monitoring

B. Tell engineers to be more careful

C. Review the incident again next year

D. Add and validate a queue-saturation alert with a documented response procedure

Correct Answer: D. Add and validate a queue-saturation alert with a documented response procedure

Question 4

A Five Whys analysis that ends with “the engineer made a mistake” is weak primarily because it:

A. uses too many questions

B. fails to examine system conditions and organisational factors

C. focuses on a technical trigger

D. requires a causal diagram

Correct Answer: B. fails to examine system conditions and organisational factors

Question 5

Which action is principally a containment control?

A. Progressive rollout that limits exposure during deployment

B. Writing a historical incident summary

C. Assigning a problem owner

D. Replacing an outdated architecture diagram

Correct Answer: A. Progressive rollout that limits exposure during deployment

Question 6

What is a central concern of systems-theoretic analysis?

A. Finding the single employee closest to the failure

B. Counting the number of alerts generated

C. Examining failures of control, constraints, feedback, and system interactions

D. Replacing all human decisions with automation

Correct Answer: C. Examining failures of control, constraints, feedback, and system interactions

Question 7

Which evidence best supports closing a corrective-action item?

A. The ticket status was changed to completed

B. The action was discussed in a meeting

C. The owner confirmed that implementation was easy

D. A test or exercise demonstrated that the intended risk reduction was achieved

Correct Answer: D. A test or exercise demonstrated that the intended risk reduction was achieved

Question 8

Why is a rising number of recorded problems not necessarily evidence of declining reliability?

A. Problem records are unrelated to incidents

B. Improved detection and reporting can increase the number of recorded problems

C. Only security teams may record problems

D. Problem records automatically close after an incident

Correct Answer: B. Improved detection and reporting can increase the number of recorded problems

Question 9

In the illustrative checkout case, why was the deployment considered a trigger rather than a complete explanation?

A. The deployment had no effect on the service

B. The incident occurred before deployment

C. Other conditions, including inadequate workload testing and weak saturation detection, enabled the impact

D. The payment provider was legally responsible

Correct Answer: C. Other conditions, including inadequate workload testing and weak saturation detection, enabled the impact

Question 10

Which practice best supports organisational learning while protecting professional integrity?

A. Publish all logs without restriction

B. Suppress uncertain findings until absolute certainty is achieved

C. Separate factual evidence from hypotheses and manage sensitive information appropriately

D. Assign disciplinary responsibility before interviewing responders

Correct Answer: C. Separate factual evidence from hypotheses and manage sensitive information appropriately