6. Statistical Quality and Performance Analytics

Introduction and context

Total Quality Management depends on disciplined learning from evidence. Experience, intuition and managerial judgement remain valuable, but they are unreliable when separated from data. Statistical quality and performance analytics provide the methods required to describe how a process is performing, distinguish routine variation from unusual variation, establish credible baselines and evaluate whether improvement has actually occurred.

In an improvement project, statistical analysis is not an isolated technical activity. It supports the Define, Measure, Analyse, Improve and Control logic of approaches such as DMAIC, while also strengthening PDCA and evidence-based management. The purpose is not to make every manager a mathematical specialist. It is to enable teams to ask better questions: What is the current level of performance? How much variation exists? Which causes matter most? Is an observed change genuine or random? Can the process consistently meet customer and specification requirements?

A useful distinction is between a process measure and a performance measure. A process measure describes an activity or intermediate condition, such as waiting time or first-pass yield. A performance measure describes the resulting outcome, such as delivery reliability, complaint rate or customer retention. Both require operational definitions, reliable data and appropriate analysis.

Learning objectives

By the end of this chapter, students should be able to:

1. Distinguish populations, samples, parameters, statistics and types of variation.

2. Select and interpret descriptive statistics and visualizations for quality data.

3. Apply Pareto analysis, probability concepts and confidence intervals.

4. Formulate and interpret hypothesis tests without confusing statistical and practical significance.

5. Use correlation and regression responsibly in process investigation.

6. Construct and interpret suitable control charts and process-capability indices.

7. Integrate statistical evidence into managerial decisions and improvement projects.

8. Recognize limitations, ethical risks and appropriate uses of digital analytics.

The statistical view of quality

A quality characteristic is a measurable property of a product, service or process. It may be continuous, such as cycle time, temperature or weight, or discrete, such as conforming/nonconforming, number of complaints or defects per transaction. Data may also be classified as nominal, ordinal, interval or ratio. The scale determines which summaries and tests are meaningful.

A population is the complete set of units of interest; a sample is the subset actually observed. A population parameter, such as the true mean delivery time, is usually unknown. A sample statistic, such as the sample mean, is used to estimate it. Sampling introduces uncertainty, while measurement systems introduce another source of uncertainty. Consequently, a precise calculation from poor or biased data is not useful evidence.

Variation has several forms. Common-cause variation is inherent in the current system and tends to produce a stable pattern over time. Special-cause variation arises from an identifiable change or unusual event, such as a machine fault, incorrect material or staffing disruption. Tampering occurs when managers react to routine fluctuation as though it were a special cause, often increasing rather than reducing variation. Statistical analysis helps determine whether intervention is justified.

Before analysis, the team should define the unit of analysis, collection period, inclusion and exclusion rules, measurement method, sampling plan and data owner. An operational definition—for example, “delivery delay means arrival more than 15 minutes after the promised time”—prevents different observers from recording different phenomena.

Describing the baseline

The arithmetic mean is the sum of observations divided by their number. It is efficient for approximately symmetric data but sensitive to extreme values. The median is the middle observation after ordering and is more robust for skewed data. The mode identifies the most frequent category or value. The range is the maximum minus the minimum, while the variance and standard deviation quantify dispersion around the mean. The interquartile range, the difference between the third and first quartiles, is particularly useful when outliers or skewness are present.

The coefficient of variation, standard deviation divided by the mean, permits comparison of relative variability when values are positive and the ratio is meaningful. Percentiles are useful for service measures: the 95th percentile of waiting time indicates a level below which approximately 95 per cent of observations fall. It should not be confused with an average.

Descriptive statistics should be read together. A mean waiting time of six minutes does not reveal whether most customers wait six minutes or whether a small number wait an hour. A box plot, histogram or time plot can expose that distinction.

Visual investigation

A histogram displays the distribution of continuous data and may reveal skewness, multiple populations or unusual values. A box plot summarizes the median, quartiles and potential outliers across groups. A run chart plots observations in time order and is often the first indication of a trend, shift or cycle. A scatter diagram examines the relationship between two variables. A bar chart is suitable for counts or categories, while a Pareto chart orders categories from largest to smallest and adds a cumulative-percentage line.

Time order is essential. Randomly rearranging observations can conceal a process shift. Graphs must include clear titles, units, denominators and relevant stratification such as product, shift, location or supplier. Truncated axes and selective time periods can exaggerate differences and are therefore misleading.

Pareto analysis

Pareto analysis applies the practical idea that a relatively small number of categories may account for a large share of an effect. It is not a law requiring an exact 80:20 ratio. The analyst counts defects, complaints, delays or costs by category, orders them by descending impact and calculates cumulative contribution. The result helps a team prioritize investigation, but it does not prove that the largest category has the most important root cause. Severity, recurrence, detectability and strategic importance may justify attention to a smaller category.

Probability and statistical inference

Probability expresses uncertainty on a scale from zero to one. For mutually exclusive events, the probability of either event is the sum of their probabilities. For independent events, the probability of both is the product of their probabilities. Conditional probability, written as P(A|B), describes the likelihood of A given that B has occurred. Confusing P(A|B) with P(B|A) is a common analytical error.

The binomial model is appropriate for a fixed number of independent trials with two outcomes and a constant probability of success. The Poisson model is often used for counts of events in a fixed space or time interval when events occur independently at a stable average rate. The normal distribution is a useful model for many continuous measurements, although normality should not be assumed automatically.

The central limit theorem explains why sample means often become approximately normally distributed as sample size increases, even when individual observations are not normal, subject to appropriate conditions. This supports estimation and testing, but it does not correct for biased sampling, dependence or poor measurement.

Confidence intervals

A confidence interval gives a range of plausible values for a population parameter, together with a stated confidence level. A 95 per cent confidence interval is produced by a method that would capture the true parameter in approximately 95 per cent of repeated samples under the method’s assumptions. It is incorrect to say that there is a 95 per cent probability that a fixed parameter lies in one already-calculated interval.

Intervals for a mean, proportion or difference between means are generally more informative than a point estimate alone. Width depends on variation, sample size and confidence level: larger samples usually narrow the interval, whereas higher confidence usually widens it. Managers should ask whether the interval is narrow enough to support a decision, not merely whether it excludes zero.

Hypothesis testing and managerial decisions

A hypothesis test begins with a null hypothesis, usually representing no difference or no effect, and an alternative hypothesis representing the claim under investigation. The analyst selects a significance level, commonly 0.05, calculates a test statistic and obtains a p-value. The p-value is the probability, assuming the null hypothesis is true, of observing results at least as extreme as those obtained. It is not the probability that the null hypothesis is true and is not a measure of practical importance.

Depending on the data and design, useful procedures include a one-sample test, a comparison of two means or proportions, paired analysis for before-and-after measurements, analysis of variance for several means, and chi-square procedures for categorical data. A paired test is appropriate when the same units are measured under two conditions; treating paired observations as independent wastes information and may distort conclusions.

There are two broad decision errors. A Type I error rejects a true null hypothesis; a Type II error fails to reject a false null hypothesis. Statistical power, the probability of detecting a specified effect, increases with sample size, effect size and lower variation, but is affected by the chosen significance level. Sound analysis reports the estimated effect and confidence interval as well as the p-value.

Statistical significance can result from a trivial improvement in a very large sample. Conversely, a practically valuable improvement may fail to reach significance in a small sample. Quality decisions should therefore combine statistical evidence with customer requirements, cost, risk and operational feasibility.

Association, prediction and regression

Correlation measures the strength and direction of linear association between two quantitative variables. Its value ranges from -1 to +1. A value near zero indicates little linear association, but does not rule out a nonlinear relationship. Correlation does not establish causation. A third variable, reverse causality, common trends or selection bias may explain an association.

Simple linear regression models an outcome as a function of a predictor: y = a + bx + error. The slope estimates the expected change in y for a one-unit change in x, within the observed range and under the model assumptions. Multiple regression permits several predictors, but correlated predictors can make individual coefficients unstable. R-squared describes the proportion of observed variation explained by the model in the sample; it is not proof of causation or predictive adequacy.

Residual plots should be examined for nonlinearity, unequal variance, dependence and unusual observations. Extrapolation beyond the data range is risky. Regression is valuable for prediction and prioritization, but causal claims generally require stronger designs, such as controlled experiments, natural experiments or carefully justified longitudinal analysis.

Control charts: seeing stability over time

A control chart combines a time-ordered sequence of measurements with a centre line and statistically calculated upper and lower control limits. The limits describe expected process variation, not customer specifications. A point beyond a limit, or a non-random pattern such as a sustained run on one side of the centre line, suggests a special cause requiring investigation.

Variable charts include the X-bar and R chart for subgroup means and ranges, and the Individuals-Moving Range chart for individual observations. Attribute charts include the p chart for the fraction nonconforming, np chart for the number nonconforming when sample size is constant, c chart for defect counts with constant opportunity, and u chart for defects per unit when opportunity varies.

The chart type must match the data structure. A chart should be established using a rational sampling subgroup: observations produced under closely similar conditions so that within-subgroup variation represents short-term common-cause variation. Limits should not be repeatedly recalculated merely to make points appear in control. When a special cause is identified and corrected, the rationale for revised limits should be documented.

A stable process is not necessarily a capable process. Stability asks whether the process is predictable; capability asks whether its predictable variation fits within specification limits. Control limits come from process behaviour, whereas specification limits come from customers, design or regulation.

Process capability and performance

For a continuous, stable and approximately normal process with lower and upper specification limits, capability indices summarize the relationship between process spread, location and specifications. The potential capability index is

Cp = (USL − LSL) / (6σ).

It compares specification width with six standard deviations but ignores centring. The centred capability index is

Cpk = min[(USL − μ)/(3σ), (μ − LSL)/(3σ)].

If the mean moves toward one specification limit, Cpk falls even when spread remains unchanged. Thus Cp can be high while Cpk is modest. A process with Cpk below Cp is off-centre. Thresholds such as 1.33 are common organizational conventions, not universal laws; the required level depends on risk, industry and customer expectations.

For non-normal data, capability calculations based on normality can be misleading. Possible responses include transforming data, fitting an appropriate distribution, using percentile-based methods or applying nonparametric approaches. Capability should be calculated only after verifying measurement adequacy and statistical stability. Pp and Ppk are performance indices based on overall, rather than within-subgroup, variation and may reveal long-term instability.

An integrated analytical workflow

A disciplined workflow links analysis to decisions:

1. Translate the customer or business problem into a measurable outcome.

2. Define the population, operational terms, specifications and data-collection plan.

3. Validate the measurement system for repeatability, reproducibility, bias and resolution.

4. Explore distributions, missing values, outliers, strata and time order.

5. Establish baseline centre, spread, defect level and process stability.

6. Use Pareto, stratification and process knowledge to generate possible causes.

7. Test relationships or improvement effects using suitable designs and methods.

8. Evaluate capability only when the process is stable and specifications are meaningful.

9. Communicate uncertainty, assumptions and practical implications.

10. Embed successful changes in control plans, visual management and ongoing monitoring.

Illustrative case: reducing medication-dispensing delays

Background

The following is an illustrative case. A hospital pharmacy receives complaints that outpatient prescriptions are not ready at the promised collection time. The improvement team defines the outcome as elapsed minutes from prescription acceptance to readiness and sets a service specification of no more than 30 minutes.

Situation and analysis

For four weeks, 480 prescriptions are recorded in time order. The mean is 34.2 minutes, the median 29 minutes and the 95th percentile 71 minutes. The histogram is strongly right-skewed, showing that a minority of prescriptions experience very long delays. A run chart reveals a shift during the afternoon period. A Pareto chart of delay reasons shows that missing stock, insurance clarification and pharmacist verification account for most delayed prescriptions, although the categories have different levels of severity.

An individuals chart indicates several points above the upper control limit, associated with stock searches and system downtime. Excluding no observations merely because they are inconvenient, the team investigates each signal. The measurement definition is checked against electronic timestamps and is found to be consistent. Capability indices are not initially used because the process is unstable and the distribution is non-normal.

Approach and solution

The team separates stock-related prescriptions from routine prescriptions, introduces an early stock check, creates an escalation rule for insurance clarification and schedules verification capacity during the afternoon peak. A pilot is conducted for another four weeks using the same definitions. The median falls to 23 minutes and the 95th percentile to 44 minutes. A comparison of paired daily medians produces a confidence interval for the improvement that excludes zero, while the team also evaluates whether the 30-minute service objective has become operationally realistic.

The revised control chart shows no special-cause signals for routine prescriptions, but stock-related work remains variable. Rather than claiming that the entire pharmacy process is “improved,” the team reports results by stratum and continues investigation of stock availability. The control plan includes daily review of the upper percentile, a chart for routine cycle time and a trigger for stock-related delays.

Learning points

The case demonstrates why averages alone are inadequate, why time order and stratification matter, and why a Pareto chart is a prioritization tool rather than a causal proof. It also shows that statistical significance must be accompanied by service relevance, stability evidence and a plan to sustain gains.

Challenges, ethics and responsible use

Common mistakes include collecting data without an operational definition, ignoring missing records, deleting outliers without investigation, using a normal-theory method for unsuitable data, treating correlation as causation, confusing control limits with specifications and repeatedly testing until a significant result appears. Multiple comparisons increase the chance of false positives and should be managed through a pre-specified analysis plan or suitable adjustment.

Data quality has an ethical dimension. Analysts should not manipulate baselines, hide unfavourable periods, change definitions after seeing results or present a narrow confidence interval created by selective sampling. Privacy must be protected when quality data contain patient, employee or customer identifiers. Access controls, de-identification, retention rules and audit trails are essential in regulated settings.

Digital dashboards and statistical software make calculation, visualization and anomaly detection easier, but automation does not remove the need for judgement. Machine-learning models may reproduce historical bias, confuse prediction with explanation or trigger unnecessary intervention. AI-generated insights should be traceable to data, validated against operational knowledge and reviewed by accountable professionals. A dashboard should display denominator, time window, versioned definitions and uncertainty where appropriate.

Emerging practice and best-practice principles

Contemporary quality analytics increasingly combines real-time process data, integrated event logs, automated control-chart alerts and predictive maintenance. Process mining can reconstruct actual workflows and reveal rework or hidden queues. Bayesian methods can update knowledge as evidence accumulates and can be useful when prior information is important. Robust and distribution-free methods are gaining attention where data are skewed, censored or affected by outliers. These developments are valuable only when measurement governance and process understanding keep pace with technological capability.

Effective practice rests on several principles: begin with a decision rather than a fashionable tool; protect data integrity; visualize before modelling; preserve time order; distinguish stability from capability; report effect sizes and uncertainty; use subject-matter knowledge; validate assumptions; and convert analysis into a documented control response. Statistical quality is ultimately a management discipline: it enables organizations to learn reliably and improve systems rather than merely produce impressive numbers.

Chapter Summary

Statistical quality and performance analytics provide the evidence base for TQM improvement projects. Descriptive statistics and visualizations establish the centre, spread, distribution and time behaviour of performance. Pareto analysis helps prioritize categories, while probability and confidence intervals express uncertainty. Hypothesis tests evaluate evidence against a defined null hypothesis, but p-values must be interpreted alongside effect size and practical importance. Correlation and regression support investigation and prediction without automatically proving causation. Control charts distinguish predictable common-cause variation from special causes. Capability indices compare stable process performance with specifications, with Cp describing potential spread and Cpk also reflecting centring. Responsible application requires valid measurement, appropriate methods, transparent assumptions, ethical governance and integration with operational action.

Key Terms

Population: The complete set of units about which conclusions are intended.

Sample: A subset of a population used for analysis.

Parameter: A numerical characteristic of a population.

Statistic: A numerical characteristic calculated from a sample.

Common-cause variation: Variation inherent in the current process system.

Special-cause variation: Variation attributable to an unusual, identifiable event.

Operational definition: A precise rule specifying how a variable is measured or classified.

Median: The central ordered observation.

Standard deviation: A measure of dispersion around the mean.

Percentile: A value below which a specified percentage of observations falls.

Pareto analysis: Ordered analysis used to prioritize important categories of effects.

Confidence interval: An interval-estimation procedure expressing uncertainty about a parameter.

Null hypothesis: The hypothesis tested as a reference claim, often no difference.

P-value: The probability of results at least as extreme as observed if the null is true.

Statistical power: The probability of detecting a specified effect when it exists.

Correlation: A measure of linear association between two variables.

Regression: A model relating an outcome to one or more predictors.

Control chart: A time-ordered chart used to assess process stability.

Specification limits: Customer, design or regulatory limits for acceptable performance.

Cpk: A capability index reflecting both process spread and centring.

Review Questions

Short Answer Questions

1. Distinguish common-cause variation from special-cause variation.

2. Why is the median often useful for skewed quality data?

3. What does a 95 per cent confidence interval communicate?

4. State one important difference between control limits and specification limits.

5. Why does correlation not establish causation?

Long Answer Questions

1. Explain how descriptive statistics, visualization and time ordering should be combined to establish a process baseline.

2. Describe the logic of hypothesis testing and explain why statistical significance must be considered with practical significance.

3. Compare the purposes and interpretations of Cp, Cpk, Pp and Ppk.

4. Explain the selection and interpretation of variable and attribute control charts.

5. Discuss ethical and governance requirements for digital quality dashboards and AI-supported analytics.

Analytical / Application Questions

1. A process has a stable mean of 52, standard deviation of 4, and specifications of 40 and 60. Calculate Cp and Cpk and interpret the difference.

2. A service process has an average cycle time of 12 minutes, but its 95th percentile is 38 minutes. What additional analyses and managerial actions would you recommend?

3. A regression shows a strong positive association between staffing levels and complaints. Identify at least three explanations other than staffing causing complaints and propose a stronger investigation design.

22. Multiple Choice Questions with Answers

Question 1

Which measure is generally most resistant to the influence of an extreme observation?

A. Mean

B. Median

C. Variance

D. Range

Correct Answer: B. Median

Question 2

A Pareto chart is primarily used to:

A. Prove the root cause of a defect

B. Estimate a population mean

C. Prioritize categories contributing to an outcome

D. Determine whether data are normally distributed

Correct Answer: C. Prioritize categories contributing to an outcome

Question 3

What does a p-value represent?

A. The probability that the null hypothesis is true

B. The probability of results at least as extreme as observed, assuming the null is true

C. The practical importance of an improvement

D. The probability that a sample is unbiased

Correct Answer: B. The probability of results at least as extreme as observed, assuming the null is true

Question 4

A process is stable but produces output outside customer specifications. This indicates that the process is:

A. Predictable but not capable

B. Unstable but capable

C. Both unstable and incapable

D. Automatically centred

Correct Answer: A. Predictable but not capable

Question 5

Which control chart is generally appropriate for the fraction nonconforming when sample sizes may vary?

A. X-bar chart

B. R chart

C. p chart

D. c chart

Correct Answer: C. p chart

Question 6

If Cp is substantially greater than Cpk, the most likely interpretation is that:

A. The process mean is off-centre relative to specifications

B. The process has no variation

C. The data must be categorical

D. The process is necessarily unstable

Correct Answer: A. The process mean is off-centre relative to specifications

Question 7

In the pharmacy case, why were capability indices not initially emphasized?

A. Capability indices apply only to defect counts

B. The process was unstable and strongly non-normal

C. The sample contained more than 100 observations

D. The specification limit was too wide

Correct Answer: B. The process was unstable and strongly non-normal

Question 8

Which statement about correlation is correct?

A. A high correlation proves that one variable causes the other

B. A correlation near zero rules out every nonlinear relationship

C. Correlation measures linear association and does not by itself prove causation

D. Correlation can be calculated only for categorical variables

Correct Answer: C. Correlation measures linear association and does not by itself prove causation

Question 9

Increasing sample size generally has which effect on a confidence interval, assuming other conditions remain similar?

A. It usually narrows the interval

B. It always shifts the interval upward

C. It removes all measurement error

D. It makes the confidence level zero

Correct Answer: A. It usually narrows the interval

Question 10

A manager adjusts a process after every ordinary fluctuation, even when no special-cause signal exists. This behaviour is called:

A. Stratification

B. Tampering

C. Regression

D. Blocking

Correct Answer: B. Tampering