Statistical Foundations and Process Capability

Introduction

Six Sigma and Lean Manufacturing rely on evidence about process behaviour. Descriptive statistics summarize what has happened; probability models describe uncertainty; sampling allows conclusions about a process without measuring every unit; and capability analysis compares process performance with customer or engineering requirements. These tools are not isolated calculations. Used together, they support a disciplined progression: define the characteristic, collect trustworthy data, understand variation, establish stability, evaluate capability, and improve the process.

A central distinction is between common-cause variation, which is produced by the ordinary design of a process, and special-cause variation, which arises from an identifiable and non-routine influence. Capability indices are meaningful only when the process is sufficiently stable and the measurement system is adequate.

Learning Objectives

By the end of this chapter, the reader should be able to:

• summarize data using appropriate descriptive statistics;
• explain probability distributions and sampling uncertainty;
• construct and interpret confidence intervals;
• distinguish common- and special-cause variation;
• assess process stability using control-chart logic;
• calculate and interpret C_p, C_pk, P_p, P_pk, yield, defects per million opportunities, and sigma metrics;
• recognize the assumptions and limitations of capability analysis;
• use statistical evidence to support operational decisions.

1. Data, Variation and Measurement

A quality characteristic is a measurable feature such as diameter, cycle time, tensile strength, temperature, or transaction accuracy. A unit is the item or service instance measured. A specification limit is a customer, design, or regulatory requirement. The upper and lower specification limits are denoted by USL and LSL.

Variation is unavoidable. It may result from raw materials, machine settings, operators, environment, maintenance, methods, or measurement. Before analysing process data, the analyst should verify:

1. the operational definition of the characteristic;
2. the sampling plan and time sequence;
3. calibration and resolution of the measurement system;
4. independence or dependence among observations;
5. the applicable specification limits.

A capable-looking process can be an illusion if measurement error is large. In measurement-system analysis, observed variation is often conceptualized as process variation plus measurement variation. If the gauge contributes materially to total variation, improvement decisions should begin with the measurement system.

Descriptive Statistics

For observations x₁,x₂,...,x_n, the arithmetic mean is:

The sample variance and sample standard deviation are:

The divisor n−1 provides an unbiased estimator of population variance under independent random sampling. The mean has the same units as the observations; variance has squared units; standard deviation has the original units.

The median is the middle ordered observation, and the mode is the most frequent value. The range is R=max(x_i)−min(x_i). The interquartile range is IQR=Q₃−Q₁, where Q₁ and Q₃ are the first and third quartiles. The coefficient of variation, useful for positive ratio-scale data, is CV=s/x̄, often expressed as a percentage. It should not be used when the mean is near zero or when the zero point is arbitrary.

The mean is sensitive to extreme observations, whereas the median and IQR are more robust. A histogram, box plot, run chart, and time-ordered plot should normally accompany numerical summaries. Averages without time order can conceal drift, cycles, or batches.

2. Probability Distributions and Sampling

A probability distribution assigns probabilities to possible outcomes. A discrete random variable has countable outcomes; a continuous random variable takes values on an interval. For a continuous variable, probability is represented by area under a density curve, and the probability of any exact single value is zero.

Normal Distribution

The normal distribution is described by mean μ and standard deviation σ. Its density is:

It is symmetric and useful for many measurements, especially when many small independent effects combine. It is not a universal model. Skewness, truncation, mixtures, autocorrelation, and outliers can make normal-based capability estimates misleading.

Standardization converts a normal observation to a standard normal variable:

The empirical rule states that approximately 68.27%, 95.45%, and 99.73% of a normal population lie within one, two, and three standard deviations of the mean, respectively. These percentages are approximations and apply to a normal distribution, not to every dataset.

Other useful distributions include the binomial distribution for the number of successes in a fixed number of independent trials, the Poisson distribution for counts of events in a fixed interval under suitable assumptions, and the t distribution for inference about a mean when population standard deviation is unknown. The t distribution has heavier tails than the normal distribution, with shape approaching normality as degrees of freedom increase.

Sampling Concepts

A population is the complete set of possible units; a sample is the observed subset. A parameter describes a population, while a statistic describes a sample. Random sampling reduces selection bias, but a large biased sample is still biased. In manufacturing, rational subgrouping is important: observations sampled close together in time may estimate short-term variation, while observations spread across shifts, lots, or days include additional sources.

The standard error of the sample mean, under independent observations with finite variance, is:

Because σ is usually unknown, it is estimated by s, giving s/√n. Increasing sample size reduces random uncertainty at a square-root rate; quadrupling sample size approximately halves the standard error. It does not remove systematic bias, poor measurement, or process instability.

For a sample proportion p̂=x/n, where x is the number of successes in n trials, the approximate standard error is √(p̂(1−p̂)/n) when the normal approximation is justified. For small counts or proportions near zero or one, exact or alternative interval methods are preferable.

3. Confidence Intervals and Statistical Interpretation

A confidence interval estimates a population parameter while expressing sampling uncertainty. For a mean with unknown population standard deviation, a two-sided interval is:

Here α is the error probability, t_α/2,n−1 is the t critical value with n−1 degrees of freedom, and 1−α is the confidence level. The interval is appropriate when observations are reasonably independent and the mean estimator is approximately normal, either because the population is normal or because sample size is sufficiently large.

A 95% confidence procedure means that, over repeated samples and under the model assumptions, approximately 95% of constructed intervals would contain the fixed population mean. It does not strictly mean that there is a 95% probability that a particular fixed parameter lies inside the already computed interval.

For a known population standard deviation, the t critical value is replaced by the normal critical value z_α/2. Confidence intervals quantify estimation uncertainty; they do not prove that a process is stable or capable. A narrow interval around a poor mean is still poor performance.

4. Common and Special Causes of Variation

Common causes are numerous, small, and inherent in the current process design. Examples include ordinary machine-to-machine differences, normal material variation, and routine environmental fluctuation. Special causes are assignable or unusual influences, such as a broken tool, incorrect program, contaminated material, or measurement error.

The distinction is managerial as well as statistical. Common-cause problems require changes to the process system—design, technology, standard work, inputs, or maintenance. Special-cause problems require identification, containment, and prevention of recurrence. Adjusting a stable process whenever an individual value fluctuates is tampering and can increase variation.

A control chart places data in time order with a centre line and statistically derived control limits. For individual observations under a stable normal model, limits are commonly estimated as:

For subgroup means, the standard error is smaller than individual-observation variation, so the limits must reflect subgroup size. In practice, constants and estimates depend on chart type, such as X̄–R, X̄–S, individuals–moving range, p, np, c, or u charts.

A point beyond a control limit is strong evidence of a special cause, but points within limits can also signal non-random behaviour. Long runs on one side of the centre line, trends, cycles, and unusual clustering should be investigated using an approved set of rules. Control limits are calculated from process behaviour; specification limits come from requirements. Confusing these two kinds of limits is a fundamental error.

5. Process Stability and Capability

A process is stable when its statistical behaviour is predictable over the period of interest. Capability asks whether that predictable behaviour can meet specifications. Capability analysis should normally follow this sequence:

1. define the characteristic and specifications;
2. validate the measurement system;
3. collect representative, time-ordered data;
4. investigate special causes;
5. confirm stability;
6. assess distributional assumptions;
7. calculate capability and performance indices;
8. convert results into improvement action.

Potential Capability: Cp and Cpk

For a stable, approximately normal process with within-process standard deviation σ_within, potential capability is:

This compares the specification width with the six-sigma spread. It ignores centring. The centred capability index is:

The two terms measure upper and lower capability. If the process mean is centred between symmetric limits, C_pk=C_p. If C_pk<C_p, the process is off-centre. A process may have excellent spread but fail because its mean is too close to one limit.

Overall Performance: Pp and Ppk

Using overall sample standard deviation s_overall, performance indices are:

Cp and Cpk use within-subgroup variation and are often interpreted as potential short-term capability. Pp and Ppk use total observed variation and describe longer-term performance. If P_pk is much lower than C_pk, variation between subgroups, time periods, shifts, or lots may be important. The exact estimation convention must be reported because software packages may use different within-standard-deviation estimators.

Capability indices do not establish causation, guarantee future quality, or replace engineering judgement. Non-normal data require transformation, distribution-free methods, or percentile-based analysis. One-sided specifications require appropriate one-sided interpretation rather than blindly applying two-sided formulas.

Worked Example

An injection-moulding process has a wall-thickness specification of 9.80 to 10.20 mm. A stable study estimates the mean as 10.06 mm, within standard deviation as 0.04 mm, and overall standard deviation as 0.05 mm.

The potential short-term capability is:

The upper and lower short-term indices are:

Therefore C_pk=1.17. The process has adequate potential spread relative to the specification width, but it is shifted toward the upper limit.

For overall performance:

The gap between C_pk=1.17 and P_pk=0.93 indicates additional long-term variation. Management should not merely recenter the process; it should investigate time-dependent causes and reduce total variation.

6. Yield, Defects and Sigma Metrics

A defect is a nonconformance to a requirement. A defective unit contains one or more defects. If D defects are found among N units, defects per unit is DPU=D/N. If each unit has O opportunities for a defect, defects per million opportunities are:

First-pass yield is the proportion that passes without rework. Rolled throughput yield for sequential steps, assuming the relevant yields are appropriately defined, is:

Yield metrics require clear opportunity definitions. Inflating the number of opportunities can make performance appear worse, while omitting meaningful opportunities can conceal risk.

Sigma conversion is convention-dependent. For a centred normal process with short-term sigma level Z, the probability beyond an upper specification is 1−Φ(Z), where Φ is the standard normal cumulative distribution function; for two symmetric tails, the total defect probability is 2[1−Φ(Z). Six Sigma programmes often report a long-term sigma level using a 1.5-sigma shift convention. This convention is historical and should be stated explicitly. A reported sigma number without its definition, tail convention, and treatment of defects is not fully interpretable.

7. Illustrative Case: Capability of a Filling Process

Background. A beverage plant fills 500 ml containers. The regulatory and customer specification is 495–505 ml. Underfilling creates compliance risk; excessive filling creates material cost.

Problem. The plant reports 99.7% first-pass yield, but customer complaints and material consumption are increasing.

Analysis. A measurement-system check confirms acceptable resolution and repeatability. Time-ordered data reveal that the process mean shifts between day and night shifts. Within each shift, estimated standard deviation is 1.0 ml; across all shifts, overall standard deviation is 1.8 ml. The combined mean is 503.0 ml.

Using within variation:

The process is poorly centred even though its short-term spread is reasonable. With overall variation:

The overall result confirms that shift-to-shift variation is substantial.

Approach. The team stratifies data by shift, verifies dosing calibration, standardizes start-up settings, and introduces a handover checklist. Control charts are maintained separately during the stabilization period, then reviewed for a common operating model.

Outcome. The mean is moved near 500 ml, and shift differences decline. Capability is recalculated only after the process demonstrates stability over a representative production period. The case illustrates why yield alone can conceal centring and long-term variation.

Learning points. Capability is a property of a defined process under stated conditions. Stratification, time order, measurement validation, and stability analysis are prerequisites—not optional statistical refinements.

8. Managerial, Ethical and Contemporary Considerations

Managers should avoid setting arbitrary capability targets without considering safety, customer risk, cost, and process maturity. A high index based on a non-representative sample can encourage false confidence. Data exclusion must be documented with an objective rule; deleting inconvenient observations is unethical and undermines governance. Regulatory characteristics require traceability, validated methods, and controlled records.

Modern manufacturing execution systems and IoT sensors can automate data capture, stratification, and near-real-time alerts. Machine-learning models may detect nonlinear patterns or predict drift, but they do not eliminate the need for sound sampling, measurement validation, causal investigation, and human accountability. Automated alarms should be evaluated for false positives, cyber-security, access control, and model drift. Statistical process control remains valuable because its assumptions and decision rules can be made transparent.

Challenges and Limitations

• Normality tests can be overly sensitive with large samples and weak with small samples; graphical and engineering judgement are also required.
• Autocorrelation violates many standard formulas and can make apparent sample size smaller than n.
• Mixed populations can produce misleading averages and distributions.
• Capability indices are unstable with small samples and should be accompanied by sample size, period, subgrouping, and estimation method.
• Specifications may be incomplete, asymmetric, or unrelated to actual customer loss.
• A capable process can still produce defects if its distribution changes or if the measurement system fails.
• Statistical significance is not the same as practical or economic significance.

Best-Practice Checklist

Use a precise operational definition; preserve time order; validate the measurement system; distinguish specifications from control limits; investigate special causes before capability calculations; report both short- and long-term variation where relevant; examine centring and spread separately; state distributional and sampling assumptions; document exclusions; stratify by meaningful process factors; and convert statistical results into preventive action.

Chapter Summary

Statistical foundations provide the language for understanding process behaviour. Descriptive statistics summarize centre, spread, and shape, while probability distributions model uncertainty. Sampling and confidence intervals quantify how much conclusions may vary from sample to sample. Common-cause variation belongs to the ordinary process system; special-cause variation requires identification and corrective action. Control charts assess stability, whereas specification limits define acceptability.

Cp and Cpk describe potential capability using within-process variation; Pp and Ppk describe observed long-term performance using overall variation. Cp compares specification width with process spread, while Cpk also considers centring. Yield, DPU, DPMO, and sigma metrics are useful only when their definitions and opportunity counts are explicit. Capability analysis is credible only when data are representative, measurements are reliable, assumptions are examined, and the process is stable.

Key Terms

Common-cause variation: Routine variation inherent in the current process system.

Special-cause variation: Non-routine variation attributable to an identifiable influence.

Population: Complete set of units or outcomes of interest.

Sample: Subset observed to learn about a population.

Parameter: Numerical characteristic of a population.

Statistic: Numerical characteristic calculated from a sample.

Standard deviation: Measure of dispersion in the original measurement units.

Standard error: Sampling standard deviation of an estimator.

Confidence interval: Interval produced by a procedure intended to contain a parameter at a stated long-run rate.

Control limit: Statistical boundary used to assess process stability.

Specification limit: Requirement defining acceptable output.

Stability: Predictability of a process over a defined period.

Cp: Potential capability index based on specification width and within-process spread.

Cpk: Potential capability index accounting for process centring.

Pp: Overall performance index based on total observed spread.

Ppk: Overall performance index accounting for centring and total observed spread.

DPMO: Defects per million opportunities.

Rolled throughput yield: Probability of passing all sequential process steps without failure.

Sigma level: Standardized performance measure whose convention must be stated.

Review Questions

Short Answer Questions

1. Distinguish a specification limit from a control limit.
2. Why is the sample standard deviation calculated using n−1 rather than n?
3. What does a confidence level describe in repeated sampling?
4. Explain the difference between common- and special-cause variation.
5. Why can C_pk be lower than C_p?

Long Answer Questions

1. Explain a disciplined workflow for conducting a valid process capability study.
2. Compare C_p, C_pk, P_p, and P_pk, including their variation estimates and managerial interpretation.
3. Discuss the assumptions and limitations of using the normal distribution in manufacturing capability analysis.
4. Explain how control charts support, but do not replace, capability analysis.
5. Evaluate the value and risks of reporting yield, DPMO, and sigma metrics to senior management.

Analytical / Application Questions

1. A stable process has USL = 52, LSL = 48, mean = 51, and within standard deviation = 0.5. Calculate C_p and C_pk, and interpret the difference.
2. A process produces 2,400 units, each with 5 defect opportunities, and 18 defects are recorded. Calculate DPMO and state one limitation of the result.
3. A process has C_pk=1.45 but P_pk=0.92. Diagnose likely sources of the gap and propose a data-collection and improvement plan.

Multiple Choice Questions with Answers

Question 1

Which measure is most directly affected by process centring as well as spread?

A. Cp

B. Cpk

C. Specification width

D. Range

Correct Answer: B. Cpk

Question 2

A process has USL = 10, LSL = 4, and within standard deviation = 1. What is Cp?

A. 0.50

B. 1.00

C. 2.00

D. 6.00

Correct Answer: B. 1.00

Question 3

Which condition is required before capability indices are interpreted as predictable process performance?

A. The process has zero variation

B. The process is statistically stable

C. The sample mean equals the target exactly

D. Every observation is identical

Correct Answer: B. The process is statistically stable

Question 4

What is the principal purpose of a control chart?

A. To define customer specifications

B. To compare suppliers by price

C. To distinguish routine from unusual process variation

D. To calculate the median only

Correct Answer: C. To distinguish routine from unusual process variation

Question 5

If 12 defects occur in 1,000 units with 4 opportunities per unit, the DPMO is:

A. 300

B. 3,000

C. 12,000

D. 48,000

Correct Answer: B. 3,000

Question 6

Which statement about Ppk is correct?

A. It uses overall observed standard deviation and accounts for centring

B. It ignores the process mean

C. It is always equal to Cp

D. It is calculated only for attribute data

Correct Answer: A. It uses overall observed standard deviation and accounts for centring

Question 7

For independent observations, increasing sample size from 25 to 100 changes the standard error of the mean approximately by:

A. Doubling it

B. Halving it

C. Quadrupling it

D. Leaving it unchanged

Correct Answer: B. Halving it

Question 8

A point beyond a control limit is normally investigated as evidence of:

A. A specification change

B. A possible special cause

C. A larger sample size

D. A guaranteed customer defect

Correct Answer: B. A possible special cause

Question 9

In the formula C_pk=min(C_pu,C_pl), the smaller component identifies:

A. The process target

B. The more capable specification side

C. The limiting side of capability

D. The sample size

Correct Answer: C. The limiting side of capability

Question 10

Why should a sigma metric convention be reported explicitly?

A. Sigma has no relationship to probability

B. Different conventions may use different tail and shift assumptions

C. Sigma can only be used for financial data

D. Sigma is identical to the process mean

Correct Answer: B. Different conventions may use different tail and shift assumptions