Solution Design, Experimentation and Risk Management

Introduction

In Six Sigma and Lean Manufacturing, identifying a problem is only the beginning of improvement. The central managerial challenge is to convert verified causes into solutions that are technically effective, economically justified, operationally feasible, and sufficiently controlled to remain effective. This chapter addresses the Improve and Control transition of DMAIC. It combines solution generation, benchmarking, creativity, decision analysis, designed experimentation, response optimization, failure-mode risk analysis, and implementation planning.

A weak improvement project often moves directly from root-cause analysis to an attractive solution. A rigorous project instead asks four questions:

1. Which possible solutions address the verified causes?
2. Which solution or combination offers the best balance of benefit, cost, risk, and feasibility?
3. What evidence demonstrates that the proposed settings or design are robust?
4. How will the organization control the process after implementation?

These questions require both statistical reasoning and managerial judgment. Experimentation does not replace process knowledge, and risk analysis does not replace empirical validation. Together, they create a disciplined path from insight to sustained performance.

Learning Objectives

By the end of this chapter, the reader should be able to:

• generate solutions using benchmarking, creativity methods, and Lean design principles;
• prioritize alternatives using weighted decision matrices and economic reasoning;
• distinguish screening experiments from optimization experiments;
• interpret full and fractional factorial designs and interaction effects;
• use response optimization while considering multiple responses;
• apply FMEA, including severity, occurrence, and detection assessment;
• design preventive and detective controls using the hierarchy of controls;
• prepare an implementation, pilot, control, and sustainment plan;
• evaluate ethical, operational, and governance risks in improvement projects.

1. From Verified Cause to Solution Architecture

A verified root cause is not automatically a solution. For example, “excessive injection temperature variation” describes a cause of dimensional defects, but several interventions may be possible: improve heater control, reduce sensor error, revise operating windows, improve preventive maintenance, or redesign the material-handling system. Solution design therefore begins by translating causes into design requirements.

A useful requirement statement has four elements:

• Cause addressed: the variable or mechanism being changed;
• required outcome: the CTQ characteristic to be improved;
• constraints: safety, regulatory, cost, capacity, quality, and delivery requirements;
• verification method: the measure that will demonstrate effectiveness.

The distinction between a solution and a requirement is important. “Install an automated inspection system” is a proposed solution. “Detect dimensional nonconformance before packing with at least 99% sensitivity and without reducing throughput below 1,000 units per hour” is a requirement.

1.1 Benchmarking

Benchmarking is the systematic comparison of processes, practices, or performance with an internal or external reference. It may be:

• internal, comparing plants, lines, shifts, or products;
• competitive, comparing direct competitors;
• functional, comparing a similar function in another industry;
• generic, comparing a process principle such as scheduling, maintenance, or error proofing.

Benchmarking should not be treated as copying. A practice that works in one context may fail elsewhere because of different materials, equipment, skills, regulations, demand patterns, or control systems. The appropriate sequence is: identify the performance gap, understand the mechanism behind superior performance, test transferability, and adapt the practice to local conditions.

1.2 Creativity and solution generation

Creativity methods expand the solution space before alternatives are evaluated. Common methods include:

• brainstorming, which separates idea generation from criticism;
• affinity analysis, which groups ideas into themes;
• SCAMPER, which asks whether a process can be Substituted, Combined, Adapted, Modified, Put to another use, Eliminated, or Reversed;
• TRIZ, which uses contradiction-based inventive principles;
• morphological analysis, which combines alternative choices for each design dimension;
• poka-yoke design, which prevents or immediately detects error;
• value engineering, which examines required functions in relation to life-cycle cost.

Effective facilitation requires psychological safety, multidisciplinary participation, and explicit separation of divergent and convergent thinking. Ideas should be linked to verified causes rather than selected because they are fashionable or technologically impressive.

2. Selecting Solutions with Decision Matrices

A decision matrix makes assumptions visible and reduces the risk that hierarchy, enthusiasm, or familiarity will dominate selection. First, define criteria and their relative importance. Typical criteria include expected quality impact, customer value, cost, implementation time, safety, regulatory compliance, maintainability, scalability, and workforce acceptance.

For alternative j and criterion i, a weighted score may be calculated as:

where Sⱼ is the total score for alternative j, wᵢ is the weight of criterion i, rᵢⱼ is the rating of alternative j on criterion i, and m is the number of criteria. Weights should sum to 1, and ratings must use a defined scale, such as 1 to 5. The score is dimensionless unless a monetary or physical measure is used directly.

The matrix suggests that Alternative C ranks highest, but the difference is small. Sensitivity analysis should therefore be performed. If a small change in weights reverses the ranking, the decision is not robust and further evidence is required. A matrix is a decision aid, not proof of technical effectiveness.

Financial analysis may supplement the matrix. If annual benefits are B, annual operating costs are C, and initial investment is I, a simple payback period is:

This measure assumes a positive annual net benefit and does not account for the time value of money, taxes, depreciation, or benefits after the payback point. For capital-intensive decisions, discounted cash-flow methods such as net present value are more appropriate.

3. Designing a Pilot

A pilot is a controlled, limited implementation intended to test technical performance, operational effects, human factors, and risk before full deployment. A credible pilot specifies:

• scope, location, product family, and duration;
• baseline and target measures;
• treatment and comparison conditions where feasible;
• sample size or observation rule;
• acceptance criteria;
• responsibilities and escalation rules;
• rollback conditions;
• data collection and analysis methods.

The pilot should not be conducted only on an unusually favorable shift or product. If the proposed solution is intended for broad use, the pilot should expose it to realistic sources of variation, while maintaining safety and traceability.

A pilot may fail because the solution is ineffective, because implementation was incomplete, or because the measurement system cannot distinguish improvement from noise. These explanations must be separated before a decision is made.

4. Design of Experiments

Design of Experiments (DOE) is a structured method for changing process factors deliberately and observing their effects on one or more responses. Unlike one-factor-at-a-time experimentation, DOE can estimate interactions and generally obtains more information per experiment.

4.1 Experimental terminology

• Response: measured output, such as cycle time, strength, defect rate, or energy use.
• Factor: controllable input suspected of influencing the response.
• Level: a specified setting of a factor.
• Treatment: a particular combination of factor levels.
• Replication: repeated experimental runs under the same treatment.
• Randomization: assigning run order randomly to reduce bias from time-related effects.
• Blocking: grouping similar experimental units to account for known nuisance variation.
• Interaction: a condition in which the effect of one factor depends on the level of another.
• Center point: a run at the midpoint of quantitative factor levels, often used to detect curvature.

The experimental unit is the smallest unit independently assigned to a treatment. In manufacturing, confusing samples with experimental units can produce pseudoreplication. For example, ten measurements from one batch are not necessarily ten independent experimental units.

4.2 Factorial designs

For k factors at two levels each, a full factorial design requires:

where N is the number of treatment combinations and k is the number of two-level factors. Replicates increase the number of observations but do not create new combinations.

A two-factor model with coded variables x₁ and x₂ can be written as:

Here, y is the response, β₀ is the intercept, β₁ and β₂ are main-effect coefficients, β₁₂ is the interaction coefficient, and ε is random error. The coded variables are usually −1 and +1 for low and high settings. The model assumes that the selected form is adequate, errors are appropriately independent or modeled, and the response behaves approximately linearly over the studied region.

In a two-level design, the main effect of factor A can be estimated as the mean response at high A minus the mean response at low A. An interaction is present when the difference between high and low levels of A changes as factor B changes. Main effects should not be interpreted in isolation when a meaningful interaction exists.

4.3 Screening and optimization

Screening experiments identify a small number of influential factors from a larger candidate list. Full factorial designs are practical for a small number of factors; fractional factorial designs reduce runs by deliberately confounding some effects. A fractional design is useful when higher-order interactions are believed to be negligible, but aliasing must be understood before interpretation.

Optimization experiments refine the settings of important factors. Response-surface methods, such as central composite or Box–Behnken designs, add intermediate levels and permit estimation of curvature. A second-order model may be represented as:

The summations cover the selected factors and distinct factor pairs. This model is useful only within the experimental region and requires adequate data, model diagnostics, and confirmation runs.

4.4 Experimental discipline

A sound DOE includes a clear objective, a stable measurement system, defined factor ranges, randomization, suitable replication, and control of nuisance variables. Analysis should examine residuals, unusual observations, lack of fit, practical significance, and statistical uncertainty. A statistically significant effect may be too small to justify implementation; conversely, a practically important effect may require more data to establish with confidence.

Experiments involving operators, customers, or safety-critical equipment require informed participation, authorization, and protection against harm. No experiment should knowingly bypass legal safeguards or create uncontrolled hazards merely to obtain a stronger signal.

5. Response Optimization

Manufacturing decisions often involve several responses. A setting that minimizes defects may increase energy use or reduce throughput. Optimization must therefore distinguish:

• target response: a value to be achieved, such as a nominal dimension;
• smaller-is-better response: such as defect rate or scrap;
• larger-is-better response: such as strength within a safe region;
• constraint response: a response that must remain below, above, or within a limit.

A practical method is desirability analysis. Each response is converted to a desirability value dᵢ between 0 and 1, where 0 is unacceptable and 1 is fully desirable. A combined desirability can be calculated as:

where p is the number of responses and wᵢ represents the importance weight of response i. The geometric form prevents a very poor response from being completely hidden by strong performance elsewhere. The desirability functions must be specified transparently; otherwise optimization can create a false appearance of objectivity.

The predicted optimum must be confirmed experimentally. Confirmation runs should be performed at the proposed settings, preferably with independent production material and realistic operating conditions. Prediction intervals, not only point predictions, should be considered when deciding whether the solution is sufficiently robust.

6. Failure Mode and Effects Analysis

FMEA is a structured, prospective method for identifying how a process, product, or system might fail, evaluating consequences and causes, and assigning actions to reduce risk. It is most valuable before implementation, although it can also be updated using field and production data.

A process FMEA generally records the process step, requirement, failure mode, effect, cause, current prevention controls, current detection controls, ratings, recommended action, owner, and completion evidence.

Traditional FMEA uses three ratings:

• Severity (S): seriousness of the effect if failure occurs;
• Occurrence (O): likelihood or frequency of the cause or failure;
• Detection (D): likelihood that existing controls will fail to detect the problem before harm or escape. A high score commonly means poor detectability.

The traditional Risk Priority Number is:

The ratings are ordinal scales, not precise ratio measurements. Consequently, an RPN of 100 is not necessarily twice as risky as an RPN of 50. Multiplication can also allow a high-severity failure to receive a modest RPN when occurrence is low. For this reason, organizations should impose special treatment for catastrophic or regulatory risks regardless of RPN and may use action-priority logic rather than RPN alone.

A strong FMEA asks whether the control prevents the cause, detects the cause, detects the failure mode, or merely detects the effect after escape. Preventive controls are generally more powerful than downstream inspection.

6.1 Risk treatment hierarchy

Risk controls should be considered in the following order:

1. eliminate the hazardous condition or failure opportunity;
2. substitute a safer material, method, or technology;
3. use engineering controls such as interlocks, guards, sensors, or automatic limits;
4. use administrative controls such as procedures, training, scheduling, and checklists;
5. use personal protective equipment as the final layer rather than the sole control.

Error-proofing is especially valuable when it makes the incorrect action impossible or immediately evident. However, automation can introduce new failure modes, including sensor drift, software errors, cybersecurity exposure, and operator overreliance.

7. Implementation and Control Planning

Implementation should be managed as a controlled change rather than an informal instruction. A complete plan identifies deliverables, owners, timing, dependencies, resources, training, communication, validation, and change-control requirements.

A useful implementation sequence is:

1. approve the solution and risk disposition;
2. update process maps, standard work, specifications, and maintenance requirements;
3. conduct training and verify competence;
4. install or modify equipment under change control;
5. execute the pilot and review evidence;
6. release the process in stages;
7. monitor leading and lagging indicators;
8. audit adherence and update the FMEA;
9. transfer ownership to the process manager.

The control plan should connect each critical input and output to a control method, sampling rule, responsible role, reaction plan, and record. Statistical process control may be used where the data type, sampling frequency, stability assumptions, and process context justify it. Control limits are calculated from process data and are not the same as customer specification limits. A process can be statistically stable but incapable of meeting specifications, or apparently capable while unstable.

A reaction plan should specify what happens when a signal occurs: stop or contain the process, identify affected material, investigate assignable causes, correct the condition, verify recovery, and document the decision. “Retrain the operator” should not be the default response to every signal; the system should first examine equipment, materials, methods, measurement, and workload.

8. Illustrative Case: Reducing Seal-Leak Defects in a Packaging Line

Background

The following is a fictional illustrative case. A food-packaging plant experiences seal-leak defects in a thermoformed tray process. The baseline defect rate is 3.8%, while the customer requirement is below 1.0%. A measurement-system study indicates that the leak test is sufficiently consistent for improvement analysis.

Problem and analysis

A Pareto analysis identifies seal temperature, dwell time, and film moisture as likely contributors. Process observation also finds that operators adjust temperature differently after changeovers. The project team defines the CTQ as leak-test pass rate and identifies three candidate factors:

• A: seal temperature, low and high settings;
• B: dwell time, low and high settings;
• C: film moisture condition, controlled and elevated.

A full 2³ design requires eight treatment combinations. The team randomizes run order, blocks by production day, and replicates selected combinations. The analysis shows a strong effect of temperature, a moderate effect of moisture, and an important temperature–dwell interaction. High temperature improves sealing only when dwell time is sufficiently long; excessive temperature at short dwell time distorts the film.

Solution design

The team considers three alternatives:

• install automatic temperature control and revise the operating window;
• add an online moisture sensor;
• require manual inspection of every tray.

The decision matrix favors automatic temperature control plus a revised standard window because it addresses the mechanism, reduces operator dependence, and provides a preventive control. Manual inspection is rejected as the primary solution because it detects defects after value has been added and cannot reliably prevent escape.

Pilot and risk analysis

A two-week pilot is conducted on one line across three product formats. The FMEA identifies sensor drift, incorrect recipe selection, heater failure, and film moisture variation as important failure modes. Controls include recipe interlocks, sensor calibration checks, heater alarms, material storage controls, and first-off verification after changeover.

The pilot reduces the observed defect rate to 0.7%. A confirmation run on independent production material produces 0.8%, meeting the target. The team also checks throughput, energy consumption, operator workload, and false alarms. Energy use increases slightly, but remains within the approved economic limit. The control plan requires automatic recipe verification, daily sensor checks, weekly review of defect trends, and a reaction plan for any leak rate above the action threshold.

Outcome and learning points

The solution succeeds because it combines DOE evidence with engineering controls and standardized work. The experiment identifies an interaction that would likely have been missed by changing one factor at a time. The FMEA prevents the team from treating the improved average as sufficient; it addresses how the new control itself could fail. The case also demonstrates that implementation success requires confirmation, ownership transfer, and monitoring after the pilot.

9. Challenges, Governance, and Contemporary Practice

Solution projects face several recurring challenges. Teams may optimize a local measure while worsening total cost, customer experience, or environmental impact. Pilot data may be selectively reported. Operators may be excluded from design decisions even though they understand practical failure modes. Models may be overinterpreted outside their experimental region. Automated systems may create opaque decisions that are difficult to audit.

Governance therefore requires traceable assumptions, version-controlled data, documented approvals, competence of experimenters, and transparent reporting of unfavorable results. Safety, environmental, labor, and product-regulatory requirements take precedence over speed or financial benefit. Where personal or worker data are collected, access, retention, privacy, and legitimate use must be controlled.

Digital twins, industrial Internet of Things platforms, machine learning, and automated optimization can accelerate experimentation, but they do not eliminate the need for process knowledge. AI-generated recommendations should be validated against physical constraints, data quality, model drift, and explainability requirements. A model that predicts reduced defects using a factor that cannot be controlled in production is not an implementable solution. Human accountability remains essential for safety-critical and regulatory decisions.

Best practice is to combine three forms of evidence: statistical evidence from well-designed experiments, engineering evidence about mechanisms and constraints, and operational evidence from realistic pilots. The strongest solution is not necessarily the one with the largest average improvement; it is the one that achieves the required performance with acceptable variation, risk, cost, and maintainability.

Chapter Summary

Solution design converts verified causes into feasible interventions. Benchmarking supplies external knowledge, while creativity methods broaden the alternatives before evaluation. Weighted decision matrices make selection criteria and trade-offs explicit, but sensitivity analysis and technical validation are necessary when rankings are close.

DOE provides an efficient framework for studying factors, interactions, and, with response-surface methods, curvature. Full factorial designs examine all combinations of selected levels; fractional designs reduce effort but introduce aliasing. Randomization, replication, blocking, measurement-system adequacy, model diagnostics, and confirmation runs are essential to credible conclusions. Multi-response optimization should make desirability assumptions visible and should be confirmed under realistic conditions.

FMEA anticipates failure modes and guides preventive and detective controls. RPN can support prioritization but should not override severity, regulatory obligations, or engineering judgment. Implementation requires change control, training, pilot evidence, control plans, reaction plans, ownership, and periodic review. Ethical governance demands truthful reporting, protection of people and the environment, traceable data, and human accountability for automated recommendations.

Key Terms

Benchmarking: Systematic comparison with an internal or external reference process or performance level.

Blocking: Grouping similar experimental units to account for nuisance variation.

Center point: Experimental setting at the midpoint of quantitative factor levels.

Confirmation run: Experiment conducted at proposed optimum settings to verify predicted performance.

Control plan: Document linking critical characteristics to controls, sampling, ownership, and reaction actions.

Design of Experiments (DOE): Planned method for studying effects of input factors on one or more responses.

Desirability: Scaled measure of how acceptable a response value is, commonly ranging from 0 to 1.

Detection rating: FMEA assessment of the likelihood that existing controls will fail to detect a problem.

Factor: Process input deliberately varied or studied in an experiment.

Interaction: Situation in which the effect of one factor depends on another factor’s level.

Level: Specific setting of an experimental factor.

Pilot: Limited implementation used to evaluate a proposed solution before broader deployment.

Poka-yoke: Error-proofing method that prevents or immediately detects incorrect action.

Randomization: Random assignment of treatment or run order to reduce systematic bias.

Replication: Repetition of an experimental treatment using independent experimental units.

Response: Measured process or product outcome in an experiment.

Response surface: Mathematical representation of a response over a region of factor settings.

Risk Priority Number: Traditional FMEA product of severity, occurrence, and detection ratings.

Severity rating: FMEA assessment of the seriousness of a failure effect.

Standard work: Documented best-known method, sequence, and conditions for performing a task.

Review Questions

Short Answer Questions

1. Why should a verified root cause not be treated as a complete solution?
2. Distinguish internal, competitive, functional, and generic benchmarking.
3. What is the purpose of randomization in a designed experiment?
4. Why can RPN be misleading in FMEA?
5. What information should a reaction plan contain?

Long Answer Questions

1. Explain how a team should move from verified causes to a prioritized solution portfolio.
2. Compare full factorial, fractional factorial, and response-surface designs, including their appropriate uses and limitations.
3. Explain the importance of interactions in process optimization and provide a manufacturing example.
4. Describe how FMEA and the hierarchy of controls should influence solution selection.
5. Develop an implementation and control strategy for a process improvement that changes equipment and standard work.

Analytical / Application Questions

1. A process team has four candidate factors and can afford only 16 experimental runs. Design an appropriate screening strategy, identify assumptions, and explain how you would protect against misleading conclusions.
2. Two solutions have weighted decision scores of 4.12 and 4.18, but the higher-scoring solution has greater safety uncertainty. Explain what additional evidence and governance steps should precede selection.
3. Construct a process-FMEA entry for an automated inspection system, including a failure mode, effect, cause, prevention control, detection control, and recommended action. Explain why your proposed action is appropriate.

Multiple Choice Questions with Answers

Question 1

Which activity most directly converts a verified root cause into a feasible improvement alternative?

A. Defining solution requirements and constraints

B. Closing the project before testing

C. Ignoring operator knowledge

D. Replacing measurement with opinion

Correct Answer: A. Defining solution requirements and constraints

Question 2

In a weighted decision matrix, the weights should normally sum to:

A. 10

B. 100

C. 1

D. The number of alternatives

Correct Answer: C. 1

Question 3

How many treatment combinations are required for a full two-level factorial experiment with three factors? N = 2^k

A. 3

B. 6

C. 8

D. nueve

Correct Answer: C. 8

Question 4

What does a meaningful interaction indicate?

A. One factor has no measurable effect

B. The effect of one factor depends on the level of another

C. All experimental errors have been eliminated

D. The response is necessarily normally distributed

Correct Answer: B. The effect of one factor depends on the level of another

Question 5

Which practice primarily reduces bias caused by time-related changes in an experiment?

A. Randomization

B. Increasing specification limits

C. Removing replication

D. Using only one operator

Correct Answer: A. Randomization

Question 6

Traditional FMEA RPN is calculated as:

A. S + O + D

B. S × O × D

C. S ÷ O × D

D. S − O − D

Correct Answer: B. S × O × D

Question 7

Which is generally the strongest control in the hierarchy of controls?

A. Personal protective equipment

B. Additional warning signs

C. Elimination of the failure opportunity

D. Annual refresher training alone

Correct Answer: C. Elimination of the failure opportunity

Question 8

Why should an experimentally predicted optimum be confirmed?

A. To verify performance under proposed and realistic operating conditions

B. To guarantee that all future variation disappears

C. To avoid documenting the experiment

D. To replace the control plan

Correct Answer: A. To verify performance under proposed and realistic operating conditions

Question 9

In desirability analysis, a desirability value of zero generally represents:

A. The most profitable setting

B. An unacceptable response value

C. The average experimental response

D. A statistically significant effect

Correct Answer: B. An unacceptable response value

Question 10

Which statement about specification limits and control limits is correct?

A. They are always identical

B. Control limits are calculated from process behavior, whereas specification limits express requirements

C. Specification limits are calculated only from run order

D. Control limits are set by customers in every case

Correct Answer: B. Control limits are calculated from process behavior, whereas specification limits express requirements