8. Design of Experiments and Robust Quality Engineering

Introduction and Context

Quality improvement is often described as a process of identifying causes of variation and controlling them. In practice, however, several potential causes operate simultaneously. Temperature, pressure, material composition, machine speed, operator method, environmental conditions, and measurement error may interact in ways that cannot be understood by changing one factor at a time. Design of experiments (DOE) provides a structured method for learning how process inputs affect outputs, while robust quality engineering extends this logic by seeking settings that perform consistently despite unavoidable noise.

This chapter develops DOE as an analytical and managerial discipline rather than merely a collection of statistical procedures. A well-designed experiment connects a business or engineering problem to measurable responses, controlled factors, realistic operating ranges, an appropriate experimental design, and a decision rule. The objective is not simply to find a statistically significant effect. It is to identify settings that improve performance, understand important interactions, quantify uncertainty, and determine whether the proposed improvement is economically and operationally sustainable.

The chapter builds from factorial experiments to fractional-factorial screening, response-surface methodology, and Taguchi-inspired robust design. It also considers replication, randomization, blocking, analysis of variance, regression, signal-to-noise measures, confirmation experiments, digital tools, and governance issues associated with data-driven experimentation.

Learning Objectives

After studying this chapter, students should be able to:

1. Explain the logic, terminology, and principles of designed experimentation.

2. Construct and interpret full and fractional factorial designs.

3. Assess main effects, interactions, curvature, and experimental error.

4. Apply analysis of variance and regression to experimental data.

5. Use response-surface methods to optimize process settings.

6. Explain Taguchi concepts, orthogonal arrays, loss functions, and robust design.

7. Distinguish controllable factors from noise factors and incorporate uncertainty into decisions.

8. Plan, analyze, and communicate an industrial DOE, including its limitations and confirmation requirements.

From Process Variation to Experimental Knowledge

A process may be represented conceptually as

\[

Y=f(X_1,X_2,\ldots,X_k;Z)+\varepsilon,

\]

where \(Y\) is a quality response, \(X_i\) are controllable input factors, \(Z\) represents noise or environmental conditions, and \(\varepsilon\) is unexplained error. The function may be unknown, nonlinear, and interaction-rich. DOE attempts to estimate enough of this function to support a decision without conducting an impractically large number of trials.

A factor is an input believed to influence a response. Its settings are called levels. A quantitative factor may be temperature or concentration; a qualitative factor may be supplier, machine, or tool type. A response is the measured outcome, such as strength, defect rate, cycle time, yield, or customer-reported performance. A treatment is a particular combination of factor levels. A run is one execution or observation under a specified treatment.

Three principles give DOE its inferential strength. Randomization protects against systematic bias caused by time, operator sequence, material batches, or equipment warming. Replication provides information about experimental variability and improves precision. Blocking groups similar experimental units so that known nuisance variation is separated from treatment effects. These principles are complementary. Repeating a biased sequence does not remove bias, and randomization alone cannot compensate for an inadequate measurement system.

Before design begins, the response must be operationally defined. “Improve quality” is not an adequate response specification. A useful definition identifies the characteristic, unit, sampling location, measurement method, timing, and acceptance criterion. If several responses matter, they should be treated as a multiple-response problem rather than silently optimizing one measure at the expense of others.

Factorial Thinking: Main Effects and Interactions

A two-level factorial design studies factors at a low level, conventionally coded as −1, and a high level, coded as +1. A full \(2^k\) factorial contains all combinations of two levels for \(k\) factors. For three factors A, B, and C, the eight treatments permit estimation of three main effects, three two-factor interactions, and one three-factor interaction.

The main effect of A is the difference between the average response at A high and the average response at A low. An interaction exists when the effect of one factor depends on the level of another. For example, increasing pressure may improve strength at a low temperature but reduce strength at a high temperature. On an interaction plot, nonparallel lines indicate interaction; crossing lines usually signal a particularly important dependence.

Interaction interpretation is central to quality engineering. If A and B interact, it is misleading to recommend the best average setting of A without specifying B. A statistically strong main effect can also be misleading when interaction is present. In managerial terms, interactions imply that process decisions must be coordinated across departments, technologies, or operating conditions rather than optimized one variable at a time.

A factorial model for a two-factor experiment can be written as

\[

Y=\beta_0+\beta_AA+\beta_BB+\beta_{AB}AB+\varepsilon.

\]

The coefficient of an interaction term describes how the effect of one factor changes with the other. Coded variables are useful because the coefficients are comparable and the design matrix is balanced. For quantitative factors, coding also makes the intercept represent the response at the center of the experimental region.

Analysis of Variance and Model Adequacy

Analysis of variance (ANOVA) partitions total variation into components attributable to model terms and residual error. For a term, the usual F-statistic compares its mean square with the residual mean square. A small p-value indicates evidence that the term contributes variation under the assumptions of the analysis; it does not by itself establish practical importance. Effect size, confidence intervals, process capability, cost, and risk must also be examined.

A useful fitted model should be subjected to residual analysis. Residuals should be examined for nonconstant variance, nonnormality, time order, outliers, and unmodeled structure. Normal probability plots, residual-versus-fitted plots, residual-versus-run-order plots, and lack-of-fit tests are useful diagnostics. A significant lack-of-fit result suggests that the model does not adequately represent the response relationship, although a nonsignificant result does not prove that the model is correct.

The coefficient of determination, \(R^2\), measures the proportion of observed variation explained by the fitted model, but a high \(R^2\) can result from overfitting. Adjusted \(R^2\), predictive checks, residual behavior, and confirmation runs provide a more reliable basis for judgment. When multiple candidate terms are considered, hierarchical modeling is generally preferred: if an interaction is retained, its associated main effects should normally remain in the model even if their individual p-values are less impressive.

Full Factorial Designs in Practice

Suppose a molding process is being studied using mold temperature, injection pressure, and cooling time. A full \(2^3\) design requires eight treatment combinations. If each combination is replicated twice, sixteen runs are performed. Randomized run order protects against drift, while a block may be introduced if the runs must be conducted over two material lots or two shifts.

The design should be preceded by a factor-screening discussion with subject-matter experts. Factors should be controllable over safe and realistic ranges. Levels should be sufficiently separated to reveal an effect, but not so extreme that the experiment enters an abnormal failure region. A pilot study may be needed to establish feasible limits. The experiment should also include a plan for handling unexpected events, missing observations, equipment failure, and deviations from settings.

Center points are often added to a two-level factorial design. A center point uses all factors at their midpoint and provides a check for curvature. If the center response differs systematically from the average of corner responses, a first-order model may be inadequate. Center points also provide a limited estimate of pure error when replicated, but they do not replace replication of the factorial treatments.

One-factor-at-a-time experimentation is attractive because it appears simple, but it cannot estimate interactions efficiently and is vulnerable to misleading conclusions when the process is nonlinear. DOE is more informative because each run contributes to the estimation of several effects. Its efficiency is especially valuable when experiments consume expensive materials, production capacity, or time.

Fractional Factorial Designs and Sequential Experimentation

When the number of factors is large, a full factorial can become expensive. A fractional factorial design uses a carefully selected fraction of the full set of combinations. A half-fraction of a \(2^k\) design contains \(2^{k-1}\) runs. The saving is achieved through aliasing, in which certain effects are confounded with one another.

For example, in a design with defining relation \(I=ABC\), the main effect A is aliased with the interaction BC. If the interaction BC is negligible, the estimated A effect can be interpreted as a main effect; if BC is important, interpretation is ambiguous. The resolution of a design summarizes the severity of aliasing. Higher-resolution designs generally separate important low-order effects more effectively, but require more runs.

Fractional designs are most effective when supported by engineering knowledge and hierarchical assumptions. Main effects are often considered more likely to be important than high-order interactions, but this is an assumption rather than a law. Alias structures must be inspected before data are collected. Foldover designs, which add runs with selected factor signs reversed, can separate previously aliased effects.

Screening experiments are usually part of a sequential strategy. An initial fractional factorial identifies a small group of potentially important factors. A refined factorial or response-surface design then estimates curvature and interactions in the reduced region. Finally, a confirmation experiment tests the selected settings under normal operating conditions. Sequential experimentation reduces cost while preserving learning, provided that decisions at each stage are documented and uncertainty is carried forward.

Response-Surface Methodology and Optimization

After important factors have been screened, the objective often changes from “Which factors matter?” to “Where should the process operate?” Response-surface methodology (RSM) addresses this question by fitting a local approximation, commonly a second-order model:

\[

Y=\beta_0+\sum_{i=1}^{k}\beta_iX_i+\sum_{i=1}^{k}\beta_{ii}X_i^2+\sum_{i<j}\beta_{ij}X_iX_j+\varepsilon.

\]

The squared terms represent curvature, while cross-product terms represent interactions. Central composite designs and Box–Behnken designs are widely used second-order designs. A central composite design combines factorial or fractional-factorial points, center points, and axial points. Box–Behnken designs avoid combinations in which all factors are simultaneously at extreme levels and can be useful where corner conditions are unsafe or infeasible.

Optimization requires more than locating the mathematical stationary point. The predicted optimum should lie within a region where the process is physically feasible, stable, and economically acceptable. Contour plots and three-dimensional response surfaces help communicate trade-offs, but numerical optimization should be accompanied by prediction intervals and sensitivity analysis.

When several responses must be optimized, desirability functions can transform each response into a scale from undesirable to desirable, then combine them into an overall desirability measure. This approach makes preferences explicit, but the chosen weights and target priorities should be reviewed with stakeholders. A solution that maximizes a composite score may not be acceptable if it violates a critical safety or regulatory constraint.

A confirmation run is essential. The selected settings are implemented independently, preferably with fresh material and normal operators. The observed response is compared with the predicted value using a prediction interval. If the confirmation result is materially different, the model, measurement system, operating discipline, or experimental region must be reconsidered.

Taguchi and Robust Quality Engineering

Robust quality engineering focuses on reducing sensitivity to noise rather than merely optimizing average performance. Noise factors are variables that affect performance but are difficult, expensive, or undesirable to control in routine production. Examples include ambient humidity, component-to-component variation, voltage fluctuation, wear, and user behavior. A robust design seeks control-factor settings that produce acceptable performance across realistic noise conditions.

Taguchi’s quality-loss perspective emphasizes that quality loss begins when performance departs from its target, even when the product remains within specification. For a nominal-the-best characteristic, a common loss model is

\[

L(y)=k(y-T)^2,

\]

where \(T\) is the target and \(k\) converts deviation into economic loss. Larger-the-better and smaller-the-better characteristics use alternative signal-to-noise formulations. The important principle is that variation and deviation from target have economic consequences beyond the binary pass/fail boundary.

Taguchi methods commonly use orthogonal arrays to study control factors efficiently. An inner array represents control-factor combinations, while an outer array represents selected noise conditions. The signal-to-noise ratio combines location and variability into a performance measure; the preferred direction depends on the response objective. Although signal-to-noise analysis is useful for screening robust settings, it should not replace analysis of means, interactions, raw data, and practical engineering constraints.

A robust-design experiment can be interpreted through a response model containing control and noise factors. A control setting is robust when its predicted response changes little across plausible noise conditions. In some cases, a control factor may intentionally interact with a noise factor so that the overall noise sensitivity is reduced. This is a key distinction: an interaction is not always undesirable; it can be exploited to make a system less sensitive.

Robust design should be distinguished from “making the process insensitive by tightening every tolerance.” Excessive control can increase cost and reduce flexibility. The preferred solution often changes product architecture, material selection, process sequence, or feedback control so that variation has less influence at its source.

Applied Scenario: Reducing Seal Failure in a Packaging Process

Background

The following is an illustrative case. A manufacturer of sterile medical pouches has experienced intermittent seal failures during accelerated shelf-life testing. The process response is seal strength, but excessive strength can damage the pouch during opening. The engineering team therefore specifies a target range rather than simply maximizing strength.

Situation and Problem

Potential controllable factors are sealing temperature, dwell time, pressure, and film supplier. Likely noise factors include film thickness variation, ambient humidity, and machine-to-machine differences. Historical records are insufficient for distinguishing their effects because operators have adjusted several settings simultaneously.

Analysis and Experimental Approach

The team first verifies the tensile-testing method through a measurement-system study and defines a sampling plan. A two-level fractional factorial design is selected for the four controllable factors, with center points and replication. The alias structure is reviewed before execution, and runs are randomized within machine blocks. Film lots are recorded so that lot-to-lot variation can be examined rather than treated as unexplained noise.

The analysis identifies strong effects for temperature and dwell time, together with a temperature-by-film-supplier interaction. Pressure has a modest main effect but contributes to an interaction with dwell time. The center points indicate curvature, suggesting that the high temperature setting is not simply better. Residual diagnostics show no severe variance problem, but a run-order plot reveals gradual jaw heating; the analysis is revised to include the block or time-related source.

Because the response must be near a target and stable, the team conducts a response-surface experiment on temperature, dwell time, and pressure. Selected settings are evaluated under combinations of low and high humidity and representative film thicknesses. The resulting model shows that a moderate temperature paired with a slightly longer dwell time reduces sensitivity to film thickness. A high-temperature setting gives a marginally higher mean strength but is much more variable.

Solution and Confirmation

The recommended operating window is chosen using a constrained desirability analysis: seal strength must remain within the target band, failure probability must be low, and cycle time must not exceed the production requirement. A confirmation study uses new film lots, both machines, normal operators, and deliberately varied humidity. The observed mean and variability fall within the model’s prediction interval. The control plan is updated to monitor temperature calibration, dwell-time accuracy, film-lot identification, and periodic seal-strength sampling.

Outcome and Learning Points

The principal improvement is not a single “best” machine setting. It is a robust operating window supported by evidence that includes interactions, curvature, noise conditions, and measurement reliability. The case illustrates several general lessons: factor screening should precede optimization; operational constraints must enter the design; machine and time effects require blocking or randomization; and confirmation under ordinary production conditions is necessary before standardization.

Challenges, Risks, and Common Errors

The most frequent failure is treating DOE as a statistical exercise detached from process knowledge. Poorly defined responses, unsafe factor ranges, uncontrolled changes, and inadequate measurement systems can invalidate technically sophisticated analysis. A statistically significant result may be practically irrelevant, while a nonsignificant result may reflect insufficient power or excessive noise rather than absence of an effect.

Multiple testing creates a risk of false discoveries when many terms and responses are examined without a prior analytical plan. Researchers should distinguish planned hypotheses from exploratory findings and confirm important discoveries. Missing runs and protocol deviations should be documented rather than silently replaced. Outliers should not be deleted solely because they contradict expectations; their physical and measurement causes should be investigated.

Extrapolation beyond the experimental region is dangerous, particularly with response-surface models. A second-order approximation is local, not a universal law of the process. Similarly, optimization based on historical observational data may confuse correlation with causation because operating settings were not assigned randomly.

Other common mistakes include ignoring interactions, failing to replicate, confusing technical repeats with independent experimental units, using an excessively narrow factor range, and reporting only p-values. A technical repeat measures the same unit again and primarily informs measurement variation; independent replication exposes process or unit-to-unit variation. The experimental unit must therefore be defined before sample size is decided.

Governance, Ethics, and Professional Responsibility

Experimental changes can affect worker safety, product conformity, customers, and regulated populations. Safety limits and regulatory requirements must never be treated as ordinary optimization constraints that can be violated for statistical convenience. In regulated industries, protocols, raw data, deviations, software versions, approvals, and analysis decisions require traceability.

Data integrity is fundamental. Analysts should preserve the original data, record transformations, distinguish planned from post hoc analyses, and maintain an audit trail for exclusions. Automated optimization systems should not be permitted to change production settings without authorization, validation, and human accountability. When experiments involve customer data, operators, or human subjects, privacy, informed participation, and fair treatment must be addressed.

Managers should also recognize the organizational consequences of experimentation. A failed experiment is not necessarily wasted effort if it reduces uncertainty, but teams need psychological safety to report unexpected results. Incentives that reward only immediate improvement encourage selective reporting and undermine learning.

Digital Tools, Automation, and Emerging Directions

Modern DOE software can generate designs, calculate alias structures, fit models, display interaction and contour plots, and perform desirability optimization. Statistical programming environments enable reproducible scripts, version control, simulation, Bayesian modeling, and integration with manufacturing data. Digital twins and process historians can help identify realistic factor ranges and noise distributions before physical experimentation.

Machine learning can support nonlinear prediction and anomaly detection, but predictive accuracy does not automatically provide causal interpretation. A black-box model may forecast responses well while failing to explain which intervention will improve them. Hybrid approaches combine designed experiments with machine learning: DOE supplies causal information efficiently, while flexible algorithms model complex residual structure.

Sequential and adaptive experimentation are increasingly important when experiments are expensive. Bayesian optimization, for example, updates a probability model after each batch of observations and selects promising future trials. Such methods must still respect safety constraints, uncertainty, measurement quality, and operational feasibility. Automated experimentation also creates governance questions concerning model drift, cybersecurity, access control, and responsibility for decisions.

Best Practices for Professional DOE

A disciplined DOE usually follows this sequence:

1. Translate the improvement opportunity into a measurable response and decision criterion.

2. Verify the measurement system and define the experimental unit.

3. Use process knowledge to identify controllable factors, noise factors, blocks, and safe ranges.

4. Select a design appropriate to the objective: screening, interaction estimation, curvature modeling, or robustness.

5. Predefine randomization, replication, blocking, sample size, and treatment of deviations.

6. Execute the experiment with disciplined records and independent oversight where appropriate.

7. Analyze effects, interactions, residuals, practical significance, and uncertainty together.

8. Optimize within feasible operating and regulatory boundaries.

9. Conduct a confirmation experiment under realistic conditions.

10. Transfer the result into standard work, monitoring, reaction plans, and continuing review.

The final step is often neglected. An experimental optimum is not a quality improvement until it is implemented, sustained, and monitored. Control charts, process capability analysis, layered audits, and periodic revalidation connect DOE with the wider quality-management system.

Chapter Summary

Design of experiments provides an efficient framework for learning how multiple process factors affect quality responses. Full factorial designs estimate main effects and interactions, while fractional factorials support economical screening through controlled aliasing. ANOVA and regression quantify effects, but residual diagnostics, practical significance, and confirmation experiments are essential for valid decisions. Response-surface methodology models curvature and supports constrained optimization. Taguchi and robust quality engineering extend optimization by considering noise factors, target deviation, loss, and sensitivity. Effective experimentation depends on measurement quality, randomization, replication, blocking, realistic factor ranges, and disciplined governance. Digital tools and adaptive methods expand analytical capability, but they do not replace causal reasoning, engineering judgment, or professional accountability.

Key Terms

Alias structure: The pattern showing which effects are confounded in a fractional-factorial design.

Analysis of variance (ANOVA): A method for partitioning response variation among model terms and residual error.

Blocking: Grouping similar experimental units to isolate known nuisance variation.

Center point: A run conducted at the midpoint of all quantitative factor ranges to assess curvature.

Confirmation experiment: An independent test of predicted performance at selected settings.

Control factor: An input that can be deliberately set or adjusted by the process designer.

Design of experiments (DOE): A planned method for studying the effects of inputs on one or more responses.

Experimental unit: The smallest unit independently assigned to a treatment.

Factor: An input variable believed to influence a response.

Interaction: A condition in which the effect of one factor depends on the level of another factor.

Orthogonal array: A balanced design structure that permits efficient, relatively independent estimation of selected effects.

Randomization: Assigning treatments in an order that protects against systematic bias.

Replication: Independent repetition of a treatment to estimate process variability and improve precision.

Response-surface methodology: A collection of designs and models used to explore and optimize responses, usually including curvature.

Robust design: Designing products or processes to perform consistently despite noise and uncertainty.

Signal-to-noise ratio: A Taguchi-oriented measure combining response level and variability according to the response objective.

Treatment: A particular combination of factor levels applied in an experimental run.

Noise factor: A variable that influences performance but is difficult or undesirable to control routinely.

Quality loss function: A model linking deviation from target to economic or societal loss.

Review Questions

Short Answer Questions

1. Distinguish a factor, a level, a treatment, and a response in a designed experiment.

2. Why are randomization, replication, and blocking important in DOE?

3. What does an interaction between two factors mean operationally?

4. What is the purpose of adding center points to a two-level factorial design?

5. Explain the difference between a controllable factor and a noise factor.

Long Answer Questions

1. Explain how a full factorial design can reveal information that a one-factor-at-a-time approach cannot.

2. Discuss aliasing and resolution in fractional-factorial designs, including their implications for interpretation.

3. Describe the role of ANOVA, regression coefficients, residual diagnostics, and practical significance in DOE analysis.

4. Explain how response-surface methodology can be used to optimize a process with curvature and multiple responses.

5. Discuss Taguchi robust design, including loss functions, orthogonal arrays, signal-to-noise ratios, and noise-factor experimentation.

Analytical / Application Questions

1. A three-factor experiment shows a strong main effect for temperature, but the temperature-by-pressure interaction is also significant. Explain how you would interpret the result and formulate an operating recommendation.

2. An organization has eight candidate process factors but can afford only sixteen experimental runs. Propose a sequential DOE strategy and identify the assumptions and risks that must be managed.

3. A fitted response-surface model predicts an optimum outside the safe operating range and performs poorly in a confirmation run. Diagnose possible causes and recommend the next analytical and operational steps.

22. Multiple Choice Questions with Answers

Question 1

Which feature most directly allows a factorial experiment to estimate whether the effect of one factor depends on another?

A. Replication

B. Interaction terms

C. Blocking only

D. Measurement averaging

Correct Answer: B. Interaction terms

Question 2

A half-fraction of a two-level factorial design is primarily used to:

A. Eliminate all experimental error

B. Study many factors with fewer runs

C. Guarantee that all high-order interactions are estimated

D. Replace the need for randomization

Correct Answer: B. Study many factors with fewer runs

Question 3

If an experiment has defining relation I = ABC, the main effect A is aliased with:

A. B

B. C

C. AB

D. BC

Correct Answer: D. BC

Question 4

What is the main purpose of center points in a two-level factorial design?

A. To estimate operator bias only

B. To test for curvature

C. To eliminate interactions

D. To create a blocking factor

Correct Answer: B. To test for curvature

Question 5

Which observation most strongly suggests that a first-order response model is inadequate?

A. All factor levels are coded as −1 and +1

B. The center-point response differs systematically from the corner average

C. The design has been randomized

D. The response has measurement units

Correct Answer: B. The center-point response differs systematically from the corner average

Question 6

In response-surface methodology, squared terms are included primarily to represent:

A. Randomization

B. Curvature

C. Blocking

D. Aliasing

Correct Answer: B. Curvature

Question 7

A robust design seeks settings that:

A. Maximize every response regardless of cost

B. Perform consistently across realistic noise conditions

C. Remove all variation by tightening every tolerance

D. Avoid all factor interactions

Correct Answer: B. Perform consistently across realistic noise conditions

Question 8

For a nominal-the-best quality characteristic, the Taguchi loss function emphasizes:

A. Only whether the item passes specification

B. The number of experimental runs

C. Economic loss as deviation from target increases

D. The elimination of replication

Correct Answer: C. Economic loss as deviation from target increases

Question 9

Why is a confirmation experiment necessary after model-based optimization?

A. It replaces the need for residual analysis

B. It tests whether predicted performance occurs under realistic independent conditions

C. It guarantees statistical significance

D. It converts qualitative factors into quantitative factors

Correct Answer: B. It tests whether predicted performance occurs under realistic independent conditions

Question 10

A significant temperature-by-film-supplier interaction in the illustrative packaging case implies that:

A. Film supplier can be ignored when setting temperature

B. The best temperature may depend on the supplier

C. Temperature has no effect on seal strength

D. Replication is unnecessary

Correct Answer: B. The best temperature may depend on the supplier