9. Experimentation, Piloting and Risk Evaluation

Introduction and Context

Continuous improvement is not complete when a solution has been proposed. A proposed change may appear logical, yet fail when exposed to real operating conditions. It may improve one measure while damaging another, work in one department but not another, or produce an apparent benefit that is actually caused by seasonality, measurement error, or changes in demand. Experimentation provides the disciplined bridge between an improvement idea and a defensible implementation decision.

In Total Quality Management (TQM), experimentation is used to learn before committing substantial resources. Piloting tests whether a change is feasible in a limited setting. Design of experiments (DOE) investigates how input factors affect outcomes. A/B testing compares alternative versions under controlled conditions. Simulation examines system behaviour where physical experimentation is expensive or disruptive. Sensitivity analysis explores how conclusions change when assumptions vary. Risk evaluation considers what can go wrong, how serious the consequences may be, and what controls are required.

These methods should not be treated as isolated statistical techniques. They are elements of a management process that connects customer requirements, process knowledge, measurement systems, operational discipline, and organizational learning. The central question is not merely, “Did the result improve?” It is: “What evidence shows that the change caused a meaningful, sustainable, and acceptable improvement without creating disproportionate risk?”

Learning Objectives

After studying this chapter, the learner should be able to:

1. Explain the purpose and logic of experimentation within TQM and continuous improvement.

2. Distinguish pilots, controlled experiments, A/B tests, simulation studies, and sensitivity analyses.

3. Design an experiment with clear outcomes, factors, controls, sampling logic, and decision criteria.

4. Interpret statistical significance, practical significance, uncertainty, and unintended consequences.

5. Apply risk assessment methods to improvement initiatives and connect risk controls with experimental design.

6. Evaluate a pilot or experiment from managerial, ethical, operational, and governance perspectives.

7. Use digital analytics and simulation responsibly in improvement decision-making.

From Improvement Idea to Evidence-Based Decision

A useful improvement experiment begins with a decision, not with a statistical technique. The project team should first state what decision will be made after the evidence is collected. For example, the decision may be whether to implement a revised inspection procedure across all plants, retain the existing process, or conduct a second pilot. Without a decision statement, teams often collect data without knowing what level of evidence is sufficient.

The next step is to formulate a testable improvement claim. A weak claim is, “The new workflow is better.” A stronger claim is, “For standard service requests, the revised workflow will reduce median completion time by at least 15% without increasing rework or customer complaints.” This formulation identifies the population, intervention, primary outcome, expected direction, and protection measures.

A sound experiment normally includes the following logic:

• Define the problem and the operational decision.
• Identify the unit of analysis, such as a transaction, patient, machine cycle, customer, or production batch.
• Specify primary, secondary, and balancing measures.
• Establish a baseline and assess measurement-system adequacy.
• Define the intervention and the conditions under which it will operate.
• Select an appropriate comparison or control condition.
• Determine sample size, duration, allocation, and stopping rules.
• Conduct the test while controlling implementation variation.
• Analyse both outcomes and unintended effects.
• Decide whether to adopt, adapt, abandon, or extend the intervention.

The distinction between a metric and a decision rule is important. A team may observe a 6% reduction in cycle time, but this observation has meaning only in relation to the cost of implementation, the required threshold, the uncertainty around the estimate, and possible deterioration in quality.

Core Concepts and Terminology

An intervention is the change deliberately introduced into a process. A factor is an input or condition that may influence an outcome, while a level is a specific setting of that factor. The response is the measured result, such as defect rate or lead time. A control group or comparison condition represents what would have occurred without the intervention, although in operational settings it may be a matched process rather than a completely untreated group.

Randomization assigns experimental units to conditions by chance, reducing systematic differences between groups. Blocking groups similar units before comparison, for example by plant, shift, product family, or customer segment. Replication repeats experimental conditions to distinguish systematic effects from random variation. Confounding occurs when the effect of an intervention cannot be separated from another factor that changed at the same time. Contamination occurs when participants or operating units assigned to one condition are exposed to the other condition.

A pilot is a limited implementation intended to test feasibility, operational readiness, and preliminary effects. A pilot is not automatically a controlled experiment. It may generate useful learning, but if there is no comparison condition, causal conclusions are weaker. An A/B test compares two versions, usually labelled A and B, with allocation designed to make the comparison fair. Simulation uses a model to represent system behaviour and evaluate scenarios without changing the real process. Sensitivity analysis tests how results vary when important assumptions or parameters change.

A primary outcome is the main measure used to judge success. Secondary outcomes provide additional evidence. Balancing measures detect harm or trade-offs elsewhere in the system. Statistical significance indicates that an observed result would be relatively unlikely under a specified null hypothesis, given the model and assumptions. Practical significance asks whether the size of the effect matters operationally, financially, clinically, or from the customer’s perspective. A confidence interval expresses a range of plausible values for an estimated effect under repeated-sampling assumptions. Power is the probability of detecting an effect of a specified size when that effect exists.

Designing a Credible Pilot

A pilot is appropriate when the organization needs to examine feasibility, training requirements, workflow integration, user acceptance, or preliminary performance before wider deployment. Its scope should be deliberately limited, but not so artificial that the findings have no relevance to normal operations. A pilot in one highly exceptional department may demonstrate technical possibility without demonstrating generalizability.

The pilot charter should identify the process owner, intervention, participating units, time period, measures, responsibilities, risks, escalation arrangements, and decision criteria. It should also state what the pilot is not intended to establish. For example, a four-week pilot may establish that a digital form can be used reliably, but it may not establish long-term productivity effects.

Baseline data should be collected using the same definitions and measurement procedures that will be used during the pilot. Teams should verify operational definitions, missing-data rules, calibration, and data integrity. If defect recording is inconsistent before and after the change, an apparent improvement may simply reflect altered detection behaviour.

The choice of pilot site requires judgement. A typical site may offer better external validity, while a willing and capable site may reduce implementation failure. One solution is staged selection: first test in a capable site to identify design problems, then test in a more representative group before full deployment.

Pilot duration should cover relevant sources of variation, including shifts, product mixes, demand patterns, and learning effects. Very short pilots may capture enthusiasm but miss fatigue, maintenance burden, or delayed defects. Very long pilots may expose the organization to unnecessary risk before an interim review.

Design of Experiments for Quality Improvement

DOE provides a structured way to learn about cause-and-effect relationships. In a one-factor-at-a-time approach, one input is changed while others are held constant. Although simple, this approach can be inefficient and may miss interactions. An interaction exists when the effect of one factor depends on the level of another. For example, a new machine speed may reduce cycle time only when a particular cooling setting is used.

A factorial design varies multiple factors systematically. In a two-level factorial design, each factor is tested at a low and high level. A full factorial design examines every combination, while a fractional factorial design uses a carefully selected subset when the number of factors is large. Screening designs help identify the few influential factors among many candidates. Response-surface methods are used after important factors have been identified, especially when the objective is to locate an operating region that optimizes a response.

A practical DOE should begin with process knowledge. Factors should be selected because they are theoretically or operationally plausible, controllable, and safe to vary. The team must distinguish controllable factors from noise factors. Noise factors cannot always be eliminated, but they can be measured, blocked, or incorporated into a robust design.

Randomization protects against time trends and hidden influences. Replication improves the estimate of experimental variation. Blocking can account for known sources of heterogeneity. Centre points or intermediate settings may help detect curvature, where the response does not change linearly between low and high levels.

The analysis should examine main effects, interactions, residual behaviour, and the practical meaning of the fitted model. A statistically impressive model is not useful if its factor settings are impossible to operate, unsafe, unaffordable, or outside the range where data were collected. Confirmation runs under predicted optimum conditions are essential before recommending standardization.

A/B Testing in Service and Digital Processes

A/B testing is particularly useful for comparing alternative forms, interfaces, messages, workflows, or service scripts. Users or transactions are assigned to version A or version B, and outcomes are compared using a predetermined primary metric. Random allocation is generally preferable because it reduces systematic differences in user characteristics and timing.

A/B tests require attention to exposure, eligibility, interference, and novelty effects. If users see both versions, carryover may invalidate the comparison. If the intervention affects a shared queue, one group may influence the experience of another. A new interface may initially perform poorly because employees are learning it, or appear successful because attention is unusually high during launch.

Teams should avoid repeatedly checking results and stopping at the first favourable fluctuation unless a sequential testing plan has been established. Multiple outcomes and repeated subgroup analyses also increase the chance of false findings. The test plan should specify the primary outcome, minimum meaningful effect, duration, allocation ratio, exclusion rules, and stopping conditions before results are examined.

In TQM, conversion rate alone is rarely sufficient. A digital form may increase completion but also increase inaccurate submissions, complaints, or downstream rework. Customer effort, accessibility, privacy, and equity should therefore be included in the evaluation.

Simulation and Sensitivity Analysis

Simulation is valuable when real-world experimentation would interrupt production, endanger people, require excessive time, or involve rare events. Discrete-event simulation can represent queues, processing times, resources, failures, and routing rules. Monte Carlo methods repeatedly sample from specified probability distributions to estimate ranges of possible outcomes. System-dynamics models are useful for feedback, accumulation, and delayed effects.

A simulation is not evidence independent of its assumptions. Model validation asks whether the model represents the real system adequately for the intended purpose. Historical data, subject-matter review, extreme-condition tests, and comparison with known system behaviour can support validation. Verification asks whether the model has been implemented correctly according to its design.

Sensitivity analysis identifies assumptions that materially influence conclusions. Suppose a proposed staffing model appears economical under average demand. If the conclusion changes when demand variability, absenteeism, or processing-time assumptions are modestly altered, the recommendation should be presented as conditional rather than certain. Scenario analysis can compare optimistic, expected, and adverse conditions; one-way sensitivity changes one assumption at a time, while probabilistic sensitivity varies several assumptions according to distributions.

Simulation should support, not replace, operational learning. Where feasible, simulated results should be followed by a controlled field test or a limited implementation with monitoring.

Interpreting Results: Significance, Uncertainty and Consequence

An observed difference is an estimate, not a certainty. Variation arises from the process, sampling, measurement, time, and implementation. A point estimate should therefore be accompanied by an uncertainty interval or an equivalent statement of precision. The team should ask whether the interval includes effects that are practically unimportant, harmful, or highly beneficial.

Statistical significance does not establish managerial importance. With a very large sample, a negligible reduction in waiting time may be statistically significant. Conversely, a valuable effect may fail to reach conventional significance in a small pilot. Practical significance should be defined before analysis where possible. It may be expressed as a minimum reduction in defects, a maximum acceptable increase in cost, a service-level target, or a customer-impact threshold.

Teams must distinguish causation from correlation. A before-and-after comparison is vulnerable to maturation, seasonality, regression to the mean, demand changes, and concurrent initiatives. Regression to the mean is especially relevant when a pilot is selected because performance was unusually poor; some improvement may occur naturally even without intervention. Comparison groups, staggered introduction, interrupted time-series analysis, and randomized allocation can strengthen causal inference.

Unintended consequences are often revealed through balancing measures. Reducing inspection time may increase escapes. Increasing production speed may increase fatigue or equipment failures. Automating triage may reduce average handling time while disadvantaging unusual cases. A mature evaluation therefore examines quality, safety, cost, employee experience, customer experience, compliance, and distributional effects rather than relying on a single average.

Risk Evaluation as Part of Experiment Design

Risk evaluation should begin before the intervention is introduced. A conventional Failure Mode and Effects Analysis (FMEA) identifies potential failure modes, their effects, causes, existing controls, and proposed actions. Severity, occurrence, and detectability ratings may be used to prioritize attention, but the ratings are aids to judgement rather than precise probabilities.

A risk-based experimental plan links each significant risk to a control or observation. If a new formulation might create a safety-critical defect, the pilot may require additional inspection, restricted release, approval by a qualified specialist, and a stop-work trigger. If a software change could corrupt records, it may require backup, access control, rollback capability, and reconciliation checks.

Risk appetite matters. A small inconvenience may be acceptable during a service pilot, whereas even a low-probability safety risk may be unacceptable without strong safeguards. High-consequence risks should not be justified merely because the expected average benefit is large. The ALARP principle—reducing risk as low as reasonably practicable—is relevant in many safety-oriented contexts, subject to applicable law and organizational policy.

A risk register should include risk description, cause, consequence, owner, likelihood, severity, existing controls, additional controls, early-warning indicators, and contingency action. Experimental governance should define who can pause the test, who reviews adverse events, and how deviations are recorded. Pilot participants and affected customers should not be treated as invisible sources of data.

Illustrative Case: Piloting a Revised Medication-Order Verification Workflow

Background

The following is an illustrative case designed for learning. A regional hospital experienced delays in verifying medication orders during peak admission periods. A quality team proposed a digital verification queue that automatically highlighted orders with missing information and routed complex cases to a senior pharmacist.

Situation and Problem

The existing process produced variable verification times and occasional rework. The proposed system promised faster processing, but managers were concerned that speed could be achieved by overlooking unusual or clinically important information. The decision was whether to expand the system beyond the pilot ward.

Analysis and Design

The team defined median verification time as the primary outcome. Secondary outcomes included percentage of orders requiring rework and queue length. Balancing measures were medication-related near misses, pharmacist workload, and escalation frequency. The unit of analysis was a medication order, but analysis also considered clustering by pharmacist and shift.

Two comparable wards participated for six weeks. One used the revised workflow and the other retained the existing process. Orders were not individually randomized because the queue design operated at ward level; instead, the team used a matched comparison and adjusted for admission volume and order complexity. Baseline measurement showed that near-miss recording varied between wards, so reporting definitions and review procedures were standardized before the pilot.

FMEA identified failure modes including incorrect prioritization, incomplete alerts, system downtime, and excessive reliance on automated prompts. Controls included mandatory review of high-risk medicines, daily reconciliation of a sample of completed orders, downtime procedures, pharmacist training, and a predefined pause rule if near misses exceeded the agreed threshold.

Results and Interpretation

The pilot ward showed a lower median verification time and a shorter peak queue. However, the reduction was smaller on night shifts, when staffing and technical support differed. Rework initially increased because the system exposed missing information that had previously been handled informally. After training, rework declined, but escalation frequency remained higher for complex orders.

The team did not interpret the early time reduction as sufficient evidence for immediate hospital-wide rollout. The comparison suggested operational benefit, but uncertainty remained concerning rare safety events and performance under unusually high demand. A sensitivity analysis indicated that the business case weakened substantially if training time and system support were underestimated.

Approach and Decision

The hospital adopted a staged recommendation: continue the workflow in the pilot ward, extend testing to two more wards with different staffing patterns, improve night-shift support, and retain enhanced review for high-risk medicines. A dashboard was established to monitor verification time, rework, near misses, alert overrides, downtime, and workload. Expansion was conditional on meeting both speed and safety thresholds.

Learning Points

The case demonstrates that a pilot is more than a small-scale launch. It is a structured learning system. The primary outcome showed potential value, but balancing measures revealed implementation and safety issues. The absence of individual randomization limited causal certainty, yet matched comparison, standardized measurement, and staged deployment improved the decision. Most importantly, the organization defined conditions under which the intervention would be paused or modified.

Challenges, Limitations and Common Errors

The most common error is beginning with a preferred solution and designing an experiment merely to confirm it. Confirmation bias can be reduced by involving people who hold different operational perspectives and by agreeing on measures before data collection.

Other frequent problems include inadequate sample size, changing the intervention during the test without documenting the change, measuring outcomes inconsistently, ignoring missing data, and selecting only favourable time periods. Teams sometimes overload a pilot with too many outcomes, creating ambiguous conclusions. A primary outcome and a small set of meaningful balancing measures provide greater discipline.

External validity is another limitation. Results from one product, site, shift, or customer group may not generalize. Conversely, excessive standardization may make the pilot unlike the real process. The organization should explicitly identify which conditions are expected to remain constant and which require further testing.

Operational experiments can also fail because of weak adoption. If employees do not understand the intervention, the experiment may test poor implementation rather than the underlying idea. Fidelity checks should therefore establish whether the change was actually delivered as designed, while allowing justified adaptations to be recorded.

Governance, Ethics and Professional Responsibility

Experimentation involving employees, customers, patients, or citizens must respect privacy, dignity, fairness, and applicable regulation. Data should be collected only for legitimate purposes, access should be controlled, and personal information should be minimized or de-identified where possible. In safety-critical or clinical settings, formal review and informed authorization may be required.

A control group should not knowingly receive inferior treatment when an established beneficial intervention exists, unless the comparison is ethically justified and appropriately governed. Automated decisions should be checked for disparate impact across relevant groups. Transparency is important when individuals are exposed to materially different service conditions or when their data are used for evaluation.

Professionals must report unfavourable findings, protocol deviations, adverse events, and uncertainty. Suppressing inconvenient results damages both organizational learning and trust. Governance also includes version control, audit trails, reproducible analysis, and clear ownership of the final decision.

Digital Tools and Emerging Practice

Modern improvement teams use process-mining platforms, statistical software, electronic dashboards, digital twins, and automated data pipelines to support experimentation. Process mining can reveal actual workflow variation before a pilot. Digital twins can test capacity and routing scenarios. Machine-learning models may predict risk or prioritize cases, but predictive accuracy does not by itself demonstrate that a proposed intervention improves outcomes.

Automated experimentation increases the importance of guardrails. Systems should include access permissions, data-quality checks, drift monitoring, rollback procedures, and human review for high-consequence decisions. Analytics teams should distinguish exploratory analysis from confirmatory testing and preserve a record of hypotheses, exclusions, and model versions.

Emerging practice is moving toward continuous experimentation, where operational data are monitored after implementation and interventions are refined iteratively. This approach is powerful but can create perpetual change fatigue and make causal interpretation difficult. A stable reference condition, documented release versions, and periodic formal reviews remain necessary.

Best-Practice Principles for Managers and Project Teams

Effective experimentation in TQM rests on several principles. Link every test to a decision. Define success in terms of customer, process, and organizational value. Check the measurement system before trusting the result. Use comparison, randomization, blocking, or time-series logic where feasible. Select a duration that represents real variation. Combine statistical evidence with practical thresholds. Track balancing measures and rare but serious events. Make risk controls part of the experimental design rather than an afterthought. Document assumptions and deviations. Treat pilots as learning systems, not public-relations demonstrations. Finally, scale only when the intervention is effective, operable, safe, and sufficiently robust across relevant conditions.

Chapter Summary

Experimentation converts improvement proposals into evidence for action. A credible pilot defines the intervention, unit of analysis, measures, comparison, duration, responsibilities, and decision rules. DOE provides a disciplined method for studying factors, interactions, and operating conditions, while A/B testing supports controlled comparison of alternative versions. Simulation and sensitivity analysis are useful when direct experimentation is costly or risky, but their conclusions depend on model assumptions.

Interpretation requires more than statistical significance. Managers must consider effect size, uncertainty, practical significance, causality, generalizability, and unintended consequences. Risk evaluation, including FMEA, risk registers, controls, and stop rules, should be integrated into the experiment. Ethical conduct requires privacy, fairness, transparency, safety, and honest reporting. Digital tools can strengthen experimentation when supported by data governance, human oversight, and reproducibility. The appropriate outcome of a pilot may be adoption, adaptation, abandonment, or further testing.

Key Terms

A/B test: A controlled comparison between two versions of an intervention, commonly labelled A and B.

Balancing measure: A measure used to detect adverse effects or trade-offs created by an improvement.

Blocking: Grouping similar experimental units to control a known source of variation.

Confidence interval: An interval expressing plausible values for an estimated parameter under specified assumptions.

Confounding: A condition in which the effect of an intervention is mixed with the effect of another factor.

Control group: A comparison group representing conditions without the tested intervention.

Design of experiments: A systematic approach to varying factors and analysing their effects on responses.

Effect size: The magnitude of a difference or relationship produced by an intervention.

FMEA: Failure Mode and Effects Analysis, a structured method for identifying and prioritizing potential process failures.

Factor: An input or condition that may influence an experimental response.

Interaction: A situation in which the effect of one factor depends on the level of another factor.

Intervention: A deliberately introduced change intended to influence process performance.

Pilot: A limited implementation used to test feasibility, effects, and operational risks.

Practical significance: The extent to which an observed effect matters in real operational or stakeholder terms.

Randomization: Assignment to experimental conditions by chance.

Replication: Repetition of experimental conditions to estimate variability and strengthen evidence.

Sensitivity analysis: Examination of how conclusions change when assumptions or input values vary.

Simulation: Computer-based representation of a system used to explore behaviour under alternative conditions.

Statistical significance: Evidence that an observed result is unlikely under a specified null hypothesis, subject to model assumptions.

Unintended consequence: An effect of an intervention that was not part of its intended objective, whether beneficial or harmful.

Review Questions

Short Answer Questions

1. Why is a decision statement necessary before designing an experiment?

2. Distinguish a pilot from a controlled experiment.

3. What is the purpose of a balancing measure?

4. Explain the difference between statistical significance and practical significance.

5. Why are randomization and blocking used in experimental design?

Long Answer Questions

1. Explain the stages involved in designing a credible continuous-improvement pilot.

2. Discuss how factorial DOE can reveal information that a one-factor-at-a-time approach may miss.

3. Examine the role of simulation and sensitivity analysis in quality improvement decision-making.

4. Explain how FMEA and risk controls can be integrated into an experimental plan.

5. Discuss the ethical and governance responsibilities of an organization conducting operational experiments.

Analytical / Application Questions

1. A call centre introduces a new script and observes a 10% reduction in average handling time, but repeat calls increase. Design a set of primary, secondary, and balancing measures and recommend an evaluation approach.

2. A manufacturing pilot is conducted only on the day shift for two weeks and reports a defect reduction. Identify threats to validity and propose a stronger design.

3. A simulation recommends reducing staff capacity because average demand is low. Describe the sensitivity analyses and field safeguards required before implementation.

22. Multiple Choice Questions with Answers

Question 1

Which statement best describes the primary purpose of a pilot in continuous improvement?

A. To guarantee that the intervention will succeed at full scale

B. To test feasibility, preliminary effects, and operational risks in a limited setting

C. To replace the need for measurement-system assessment

D. To demonstrate statistical significance regardless of implementation conditions

Correct Answer: B. To test feasibility, preliminary effects, and operational risks in a limited setting

Question 2

A new inspection method reduces defects but increases customer waiting time beyond the service commitment. Which measure most directly reveals this trade-off?

A. A balancing measure

B. A blocking factor

C. A replication measure

D. A nuisance parameter

Correct Answer: A. A balancing measure

Question 3

What is the main advantage of randomization in an experiment?

A. It eliminates all process variation

B. It guarantees practical significance

C. It reduces systematic differences between experimental conditions

D. It removes the need for a control group in every situation

Correct Answer: C. It reduces systematic differences between experimental conditions

Question 4

A team changes machine speed and cooling setting together and observes fewer defects. Why may the result be difficult to interpret?

A. The response has been replicated

B. The factors may be confounded

C. The study contains a balancing measure

D. The result is necessarily practically insignificant

Correct Answer: B. The factors may be confounded

Question 5

Which finding most clearly indicates practical rather than merely statistical significance?

A. The p-value is below a chosen threshold, but the time saving is negligible

B. The confidence interval is narrow around a very small effect

C. The improvement exceeds the predefined operational threshold without unacceptable trade-offs

D. A large sample produces a statistically significant difference in an irrelevant metric

Correct Answer: C. The improvement exceeds the predefined operational threshold without unacceptable trade-offs

Question 6

Why should a simulation model undergo validation before being used for a major quality decision?

A. Because simulation results are independent of assumptions

B. Because validation checks whether the model adequately represents the real system for its intended purpose

C. Because validation converts simulated results into randomized evidence

D. Because validated models never require field testing

Correct Answer: B. Because validation checks whether the model adequately represents the real system for its intended purpose

Question 7

A pilot is selected because a department has unusually poor performance, and performance later improves even before the intervention is fully implemented. Which phenomenon may partly explain the improvement?

A. Regression to the mean

B. Interaction effect

C. Contamination control

D. Response-surface optimization

Correct Answer: A. Regression to the mean

Question 8

Which action best integrates risk evaluation into an experiment?

A. Review risks only after the pilot has ended

B. Remove all balancing measures to simplify analysis

C. Link identified failure modes to controls, indicators, owners, and stop rules

D. Accept high-consequence risks whenever average benefits are positive

Correct Answer: C. Link identified failure modes to controls, indicators, owners, and stop rules

Question 9

An A/B test repeatedly checks results and stops immediately when one version appears favourable, without a predefined stopping plan. What is the principal concern?

A. Increased risk of a misleading favourable result

B. Excessive blocking of experimental units

C. Failure to create any intervention

D. Automatic loss of external validity due to randomization

Correct Answer: A. Increased risk of a misleading favourable result

Question 10

In the illustrative medication-verification case, why was hospital-wide rollout not recommended immediately?

A. The new workflow produced no operational improvement

B. The primary outcome was ignored in favour of cost alone

C. Safety, shift variation, rare events, and support assumptions required further testing

D. The comparison ward used exactly the same workflow as the pilot ward

Correct Answer: C. Safety, shift variation, rare events, and support assumptions required further testing