Bayesian Probability and Probabilistic Networks

Introduction

Quality management is fundamentally an exercise in learning under uncertainty. A manufacturer may observe a failed component without knowing whether the cause was supplier variation, machine deterioration, operator error, measurement error, or an interaction among these factors. A healthcare organisation may observe an adverse event while the true causal pathway remains uncertain. A service organisation may receive a complaint and must infer whether it reflects a systemic process problem or an isolated occurrence.

Bayesian probability provides a formal framework for updating beliefs when evidence becomes available. It combines prior knowledge, observed data, and a probabilistic model to produce a posterior distribution. This makes Bayesian reasoning especially valuable in Total Quality Management (TQM), where decisions often must be made before large samples are available and where expert knowledge, historical information, process data, and causal understanding all matter.

This chapter develops Bayesian probability from Bayes’ theorem to hierarchical models and Bayesian networks. It also examines interpretation, computation, quality applications, limitations, governance, and the distinction between prediction and causation.

Learning Objectives

By the end of this chapter, the reader should be able to:

1. Explain prior, likelihood, evidence, and posterior distributions.
2. Apply Bayes’ theorem to diagnostic and quality-control problems.
3. Construct and interpret conjugate Bayesian models.
4. Distinguish credible intervals from frequentist confidence intervals.
5. Explain hierarchical models and partial pooling.
6. Represent conditional dependence using Bayesian networks.
7. Use probabilistic networks for quality learning and causal risk assessment.
8. Evaluate Bayesian models critically, including data, prior, computational, ethical, and governance issues.

1. Bayesian Reasoning as Quality Learning

A probability can be interpreted in two complementary ways. In a frequentist interpretation, probability describes long-run relative frequency under repeated sampling. In a Bayesian interpretation, probability quantifies rational uncertainty about an event, parameter, or hypothesis given available information. The Bayesian interpretation is particularly suitable when the object of interest is unknown but not inherently random—for example, the defect rate of a production line during a particular month.

Let θ denote an unknown parameter, such as a defect probability. Before observing current data, knowledge about θ is represented by a prior distribution, written p(θ). After observing data D, the updated uncertainty is the posterior distribution, p(θ∣D).

The central relationship is:

Here:

• p(θ∣D) is the posterior distribution;
• p(D∣θ) is the likelihood of the observed data under parameter value θ;
• p(θ) is the prior distribution;
• p(D) is the marginal probability, or evidence, of the data.

For continuous parameters, the evidence is obtained by integration:

The evidence ensures that the posterior integrates to one. In many parameter-estimation problems, it is sufficient to write:

The symbol ∝ means “proportional to.” This form is useful because the posterior can first be calculated up to a constant and then normalised.

The four elements of Bayesian analysis

A Bayesian analysis is therefore not simply “using a prior.” It is a complete model of uncertainty, data generation, and updating.

2. Bayes’ Theorem for Competing Hypotheses

For discrete hypotheses H₁, H₂, …, H_k, Bayes’ theorem is:

The hypotheses must be mutually exclusive and collectively exhaustive. The term P(D∣H_i) is the likelihood of evidence under hypothesis H_i, while P(H_i) is its prior probability.

Worked example: inspection screening

Suppose 2% of components are genuinely defective. An automated inspection system identifies a defective component with probability 0.95 and incorrectly flags a conforming component with probability 0.04. Let F denote a positive inspection result and D denote a genuinely defective component.

Given:

• P(D)=0.02;
• P(F∣D)=0.95;
• P(F∣D^c)=0.04;
• P(D^c)=0.98.

The probability of a positive result is:

Therefore:

Thus, a flagged component has approximately a 32.65% probability of being defective. The result is much lower than 95% because false positives among the large population of conforming components are substantial. This is the base-rate effect, a critical issue in quality screening, fraud detection, medical diagnosis, and alarm management.

3. Bayesian Updating for Quality Parameters

3.1 The binomial–Beta model

Suppose a process produces independent units, each classified as defective or conforming, with a constant defect probability θ. If y defects are observed in n units, then:

The likelihood is:

where C(n,y) is the binomial coefficient. The model assumes conditional independence of units, a common defect probability during the observation period, and correct classification of outcomes.

A Beta prior is written:

with density proportional to θ^(α−1)(1−θ)^(β−1), where α>0 and β>0. Combining the likelihood and prior gives the conjugate posterior:

The prior mean is:

The posterior mean is:

The prior can be interpreted as contributing prior information, although the “prior sample size” interpretation should be used cautiously because it depends on the model and context.

Worked example: updating a defect-rate estimate

Assume historical evidence supports a prior distribution θ~Beta(2,98). Its prior mean is:

In a new sample of 40 units, 3 are defective. The posterior is:

The posterior mean is:

The current data have increased the estimated defect rate from 2.0% to approximately 3.57%, but not to 7.5%, which would be the raw sample proportion. The posterior reflects both current evidence and historical information. With substantially more current data, the posterior would be influenced less by the prior.

A posterior interval should normally be reported as a credible interval. For example, a 95% credible interval is an interval [L,U] satisfying:

Its numerical endpoints are obtained from the Beta distribution’s quantiles. A credible interval makes a probability statement about θ conditional on the model and data. This differs from a 95% frequentist confidence interval, whose repeated-sampling interpretation concerns the procedure rather than the probability that the fixed parameter lies in the particular interval.

3.2 Predictive probability

Bayesian analysis also supports prediction. If X_new is the outcome of a future unit, then:

For a future defect under the Beta–Binomial model:

Prediction averages over parameter uncertainty rather than substituting only a single estimate. For a future batch of m units, the predictive number of defects follows a Beta–Binomial distribution:

where B(·,·) is the beta function. This formula assumes the same underlying defect probability applies to future units and that the model remains appropriate.

4. Likelihoods, Priors, and Model Construction

The likelihood is not itself a probability distribution over the parameter. It is a function of the parameter for fixed observed data. This distinction prevents a common error: treating P(D∣θ) and P(θ∣D) as interchangeable.

Prior specification should be transparent and defensible. Common approaches include:

1. Informative priors: based on reliable historical process data or validated expert knowledge.
2. Weakly informative priors: constrain implausible values while allowing the data to dominate.
3. Reference or diffuse priors: used when prior information is limited, although “non-informative” does not mean assumption-free.
4. Robust mixture priors: combine a substantive prior with a broader component to reduce sensitivity to prior misspecification.

Prior sensitivity analysis should examine how conclusions change under credible alternative priors. If managerial decisions change substantially, the prior-data conflict must be investigated rather than hidden.

A useful diagnostic is prior predictive checking. Before observing current data, simulate outcomes from the prior and likelihood and ask whether the resulting process behaviour is plausible. If a model predicts almost impossible numbers of defects, cycle times, or failures, its assumptions require review.

5. Hierarchical Bayesian Models

Quality data commonly have multiple levels: units within batches, batches within machines, machines within plants, and plants within regions. Treating all observations as independent and identically distributed can conceal meaningful variation and produce overconfident conclusions.

A hierarchical model introduces group-specific parameters connected through a population distribution. For groups j=1,…,J, a simple model is:

The group defect probabilities θ_j are not forced to be equal, but they are assumed to arise from a common distribution. The hyperparameters α and β describe the population-level distribution and may themselves receive hyperpriors.

In a normal-means formulation, useful for continuous quality measurements, one might specify:

where y_ij is measurement i in group j, μ_j is the group mean, μ is the overall mean, σ² is within-group variance, and τ² is between-group variance.

Partial pooling

Hierarchical models create partial pooling. A group estimate is influenced by its own data and by information from other groups. A small plant with few observations is shrunk toward the population mean more strongly than a large plant with abundant data. This is often preferable to:

• No pooling, which estimates every group independently and can overreact to small samples;
• Complete pooling, which assumes all groups have exactly the same parameter and can conceal real differences.

For a normal observation model with known variance, the posterior mean has the form of a weighted average:

where ȳ_j is the group sample mean and w_j depends on within-group sample size, measurement variance, and between-group variance. The exact weight depends on the specified model. The important managerial implication is that extreme results based on very little evidence should be investigated, but not automatically ranked as the worst performers.

Hierarchical models are particularly useful for supplier quality, multi-site benchmarking, reliability across product families, and repeated audits. They also support learning while preserving group-level heterogeneity.

6. Bayesian Networks

A Bayesian network is a directed acyclic graph (DAG) whose nodes represent random variables and whose directed edges represent conditional dependence according to the model. The graph is paired with conditional probability distributions.

For variables X₁,…,X_n, the joint distribution factorises as:

where Pa(X_i) denotes the parents of node X_i. The graph must contain no directed cycle. The factorisation can greatly simplify inference and make assumptions explicit.

Consider a quality-risk network with:

• Supplier Material (S);
• Machine Condition (M);
• Operator Training (O);
• Process Variation (V);
• Defect (D).

A plausible structure may include S→V, M→V, O→V, and V→D. This does not prove that the arrows are causal. It expresses a modelling hypothesis requiring domain justification and, for causal claims, suitable design or assumptions.

Conditional independence

A key benefit of a Bayesian network is its representation of conditional independence. If the model implies that supplier material and machine condition influence defects only through process variation, then they may be conditionally independent given process variation, depending on the complete graph. Conditional independence can reduce the amount of data needed and clarify which variables should be measured.

The graph structure is often interpreted using d-separation. A path can be blocked by conditioning on a non-collider, while conditioning on a collider can open an association. This is crucial in quality investigations. For example, selecting only units that triggered an alarm may create a spurious association between two otherwise independent causes if the alarm is a common effect of both causes.

Bayesian network inference

Inference calculates a probability of interest after evidence is entered. Exact inference is feasible for small networks using enumeration, variable elimination, or message passing. Large or densely connected networks may require approximate methods such as Monte Carlo simulation, Markov chain Monte Carlo, importance sampling, or variational inference.

A network can answer questions such as:

• What is the probability of a defect given a failed inspection?
• How does evidence of machine vibration change the probability of bearing failure?
• Which unobserved cause is most probable given an observed failure pattern?
• What is the predicted risk after a proposed control is introduced?

The network should distinguish observational evidence from intervention. Observing M=m is not equivalent to forcing machine condition to M=m. In a causal DAG, intervention is represented conceptually by removing incoming causal arrows into the intervened variable and evaluating the resulting distribution. This distinction prevents the common mistake of interpreting correlation as the effect of a quality intervention.

7. Causal Risk Assessment

Probabilistic networks support risk assessment by combining initiating events, intermediate conditions, barriers, and consequences. A quality-risk model may represent:

1. initiating conditions, such as supplier variation or software change;
2. process states, such as temperature drift or loss of calibration;
3. barrier performance, such as inspection sensitivity or maintenance effectiveness;
4. outcomes, such as defect escape, customer complaint, or safety event.

The model can produce a posterior risk estimate after evidence is observed. It can also support decision analysis by assigning utilities or costs to outcomes. If action a has uncertain consequences, its expected utility is:

where s indexes possible states, U(a,s) is the utility of action a in state s, and P(s∣D,a) is the relevant predictive probability. A rational decision rule selects the action with the highest expected utility, subject to constraints, safety requirements, and ethical obligations.

Expected value alone may be insufficient where catastrophic outcomes are possible. Organisations may impose precautionary thresholds, require independent review, or prohibit actions whose low-probability consequences violate safety standards.

8. Illustrative Case: Bayesian Learning in a Multi-Plant Assembly Process

Background

The following is a fictional illustrative case. A company operates three plants producing the same electromechanical assembly. Management wishes to compare defect rates and decide whether Plant C requires immediate intervention. Each plant has different production volumes and inspection histories.

Problem

During one month, the observed results are:

Raw defect rates are 1.8%, 4.0%, and 6.25%, respectively. Ranking plants by these percentages may be misleading because Plant C has a small sample and Plant B may operate under different product-mix conditions.

Analysis

A hierarchical Bayesian model is specified:

The prior hyperparameters are estimated from stable historical data and checked through prior predictive simulation. A separate covariate for product family is included because product mix affects defect opportunity. The model therefore does not assume that every observed difference is a plant effect.

Posterior estimates are summarised by posterior means and 95% credible intervals. Plant C’s posterior mean remains higher than Plant A’s, but its interval is wide. The probability that Plant C’s defect rate exceeds a management threshold of 5% is calculated from its posterior distribution. Plant B’s posterior distribution overlaps Plant C’s, indicating that the evidence does not justify a categorical conclusion that C is uniquely poor.

A Bayesian network is then constructed for Plant C. Nodes represent supplier material condition, machine calibration, operator training, process variation, inspection result, and final defect. Historical failure investigations provide initial conditional probabilities; current vibration readings and supplier test results are entered as evidence. The posterior probability of calibration drift increases materially after the vibration evidence is observed.

Approach and solution

Management adopts a staged response:

1. Verify the calibration system and inspect the relevant machine.
2. Increase short-term sampling for the affected product family.
3. Conduct a focused supplier-material investigation.
4. Avoid permanently labelling Plant C as poor until product mix and sample-size effects are resolved.
5. Re-estimate the model after additional evidence arrives.

This combines Bayesian estimation, network-based diagnosis, and operational controls. The decision is not based solely on the most probable cause; it also considers detection cost, safety impact, reversibility, and the consequences of delayed action.

Outcome

The calibration check identifies intermittent sensor drift. After recalibration and a controlled follow-up sample, the posterior probability that the defect rate exceeds 5% declines substantially. The supplier hypothesis is not eliminated, but its posterior probability is lower than initially suspected.

Learning points

• Small samples can produce unstable rankings.
• Hierarchical models support fairer comparisons through partial pooling.
• A Bayesian network makes diagnostic assumptions visible.
• Evidence should update, not replace, engineering judgement.
• Corrective action should reflect both probability and consequence.

9. Computation, Validation, and Reporting

Closed-form conjugate models are useful for teaching and simple applications, but advanced models often require computation. A Bayesian workflow commonly includes:

1. translating the process into variables and assumptions;
2. specifying priors and likelihoods;
3. checking prior predictive behaviour;
4. fitting the model using exact or approximate inference;
5. checking convergence and computational diagnostics;
6. conducting posterior predictive checks;
7. testing sensitivity to priors, missing data, and alternative structures;
8. communicating uncertainty and decisions.

For Markov chain Monte Carlo, convergence diagnostics may include effective sample size, trace plots, and statistics such as Ŕ̂. A value close to 1 is generally sought, but no single diagnostic proves correctness. Posterior predictive checking compares replicated data generated from the fitted model with observed data. A model that reproduces the mean but fails to reproduce extreme failures may be inadequate for risk management.

Reports should state the data period, inclusion rules, measurement system, prior rationale, model assumptions, missing-data treatment, computational method, convergence evidence, uncertainty intervals, and sensitivity results. A single posterior mean without its uncertainty is not an adequate quality report.

10. Limitations, Ethics, and Governance

Model uncertainty

A posterior distribution conditional on a model does not automatically account for uncertainty about the model itself. Alternative likelihoods, network structures, measurement-error assumptions, and missing-data mechanisms may produce different conclusions. Model averaging or explicit scenario analysis may be appropriate.

Prior influence and power

Expert priors can encode valuable knowledge, but they can also institutionalise bias or favour incumbent suppliers. Priors should be documented, reviewed, and subjected to sensitivity analysis. “Letting the data speak” is not a solution when data are sparse, biased, or selectively collected.

Measurement and selection bias

Inspection results may be imperfect. If the measurement process is ignored, the posterior defect rate may be too certain or systematically wrong. Similarly, data generated only after complaints or alarms are not representative of routine operations. Selection mechanisms should be modelled where material.

Privacy and fairness

Quality analytics may involve employees, customers, suppliers, or patients. Data minimisation, access controls, audit trails, and lawful processing are necessary. If Bayesian risk scores influence personnel or supplier decisions, the organisation should assess disparate impact, provide review mechanisms, and avoid treating probabilistic rankings as unquestionable facts.

Safety and accountability

A low posterior probability of failure does not justify ignoring a severe hazard if evidence is incomplete or consequences are unacceptable. Bayesian evidence should support, not replace, mandatory safety controls, regulatory requirements, professional standards, and accountable human judgement.

11. Contemporary Developments and Best Practices

Modern Bayesian quality systems increasingly combine sensor streams, reliability data, maintenance records, text from nonconformance reports, and expert assessments. Dynamic Bayesian networks extend ordinary networks over time and can model degradation, maintenance, and recurrent failures. Bayesian state-space models support online updating when process conditions evolve.

Machine learning systems can provide flexible predictive components, while Bayesian methods contribute uncertainty quantification, prior knowledge, and decision transparency. However, a highly predictive model is not automatically causal. Explainable AI tools may describe model behaviour but cannot establish that an intervention will produce a predicted change.

Best practice includes:

• define the decision before selecting the model;
• distinguish prediction, diagnosis, and causal intervention;
• use a measurement-system analysis before modelling defect data;
• represent process hierarchy explicitly;
• justify priors and perform sensitivity analysis;
• validate models using engineering knowledge and posterior predictive checks;
• report distributions, not only point estimates;
• document data provenance and model revisions;
• use independent review for high-consequence decisions;
• monitor model performance after deployment.

Chapter Summary

Bayesian probability provides a coherent framework for updating uncertainty using prior knowledge and observed evidence. Bayes’ theorem combines prior probabilities with likelihoods to produce posterior probabilities. In quality applications, the binomial–Beta model offers a transparent method for updating defect-rate beliefs and generating predictive probabilities. Credible intervals express uncertainty about parameters conditional on the model and data.

Hierarchical Bayesian models are appropriate for nested quality data and enable partial pooling, preventing both overreaction to small samples and the concealment of genuine group differences. Bayesian networks represent conditional dependence through directed acyclic graphs and factorise joint probability distributions. They support diagnosis, risk assessment, and, when causal assumptions are justified, intervention analysis.

Effective Bayesian quality management requires more than computation. It requires defensible priors, valid measurement, transparent assumptions, predictive checking, sensitivity analysis, ethical governance, and careful distinction between association and causation. Bayesian models should inform accountable decisions rather than become opaque substitutes for engineering and managerial judgement.

Key Terms

Prior distribution: Probability distribution representing uncertainty before current data are observed.

Likelihood: Function describing how compatible observed data are with parameter values.

Posterior distribution: Updated distribution after combining prior information with data.

Evidence: Marginal probability of the observed data under the model.

Bayes’ theorem: Rule for updating probabilities using prior probabilities and likelihoods.

Base-rate effect: Influence of the prevalence of an event on the interpretation of evidence.

Conjugate prior: Prior distribution that produces a posterior in the same distributional family.

Beta distribution: Distribution on [0,1] commonly used for probabilities and defect rates.

Predictive distribution: Distribution of future observations after averaging over posterior uncertainty.

Credible interval: Bayesian interval containing a specified posterior probability of the parameter.

Hierarchical model: Model containing parameters at multiple levels of organisation.

Partial pooling: Estimation method that combines group-specific and population-level information.

Hyperparameter: Parameter governing a prior distribution for lower-level parameters.

Bayesian network: Directed acyclic graph with probability distributions defining a joint model.

Conditional independence: Independence between variables given specified information.

D-separation: Graphical criterion for identifying conditional independence in a DAG.

Collider: Node receiving arrows from two or more variables on a path.

Causal intervention: Deliberate setting or manipulation of a variable, distinct from observation.

Posterior predictive checking: Evaluation of a model by comparing observed data with data simulated from its posterior.

Model uncertainty: Uncertainty about the correct structure, assumptions, or probability model.

Review Questions

Short Answer Questions

1. Define prior distribution, likelihood, and posterior distribution in a quality-control context.
2. Why can a positive inspection result have a relatively low probability of indicating a true defect?
3. State the posterior distribution for a binomial likelihood with a Beta prior.
4. What is partial pooling, and why is it useful in multi-plant quality comparison?
5. Distinguish an observational probability from a causal intervention probability.

Long Answer Questions

1. Explain Bayes’ theorem and discuss each of its four components using a quality-screening example.
2. Derive the Beta–Binomial posterior and explain the assumptions required for its use.
3. Compare no pooling, complete pooling, and partial pooling in hierarchical quality models.
4. Explain how Bayesian networks represent joint probability distributions and conditional independence.
5. Discuss ethical, governance, and measurement issues that can undermine Bayesian quality decisions.

Analytical / Application Questions

1. A process has a Beta(3,97) prior defect rate. In a sample of 60 units, 4 defects are observed. Calculate the posterior distribution and posterior mean, and explain how the result differs from the raw sample proportion.
2. Design a Bayesian network for diagnosing recurring defects in a machining process. Identify at least five nodes, explain the proposed arrows, and distinguish diagnostic inference from causal intervention.
3. Two plants have different sample sizes and observed defect rates. Propose a hierarchical Bayesian analysis, identify relevant covariates and sensitivity checks, and explain how management should act if the smaller plant has the higher raw rate but substantial posterior uncertainty.

Multiple Choice Questions with Answers

Question 1

In Bayes’ theorem, what does P(D∣θ) represent?

A. The posterior probability of θ

B. The likelihood of the data given θ

C. The prior probability of the data

D. The marginal probability of θ

Correct Answer: B. The likelihood of the data given θ

Question 2

A process has prior θ~Beta(2,98). If 3 defects are observed in 40 units, what is the posterior distribution?

A. Beta(2,138)

B. Beta(5,135)

C. Beta(5,138)

D. Beta(42,98)

Correct Answer: B. Beta(5,135)

Question 3

In the inspection example, why is the probability that a flagged component is truly defective less than 95%?

A. The test has no sensitivity

B. The defect base rate is low and false positives occur

C. The prior probability is necessarily zero

D. Bayesian updating ignores test evidence

Correct Answer: B. The defect base rate is low and false positives occur

Question 4

What is the posterior mean of θ when θ∣y~Beta(5,135)?

A. 0.0200

B. 0.0357

C. 0.0500

D. 0.0750

Correct Answer: B. 0.0357

Question 5

What is the principal benefit of partial pooling in a hierarchical model?

A. It forces all group parameters to be identical

B. It ignores population-level information

C. It balances group data with information from other groups

D. It eliminates uncertainty from small samples

Correct Answer: C. It balances group data with information from other groups

Question 6

Which condition is required for a standard Bayesian network graph?

A. The graph must contain at least one directed cycle

B. The graph must be a directed acyclic graph

C. Every node must have exactly two parents

D. All variables must be continuous

Correct Answer: B. The graph must be a directed acyclic graph

Question 7

What does a Bayesian network factorisation multiply together?

A. Marginal means only

B. Prior variances only

C. Each variable’s conditional distribution given its parents

D. Only the probabilities of observed nodes

Correct Answer: C. Each variable’s conditional distribution given its parents

Question 8

Which statement correctly distinguishes a credible interval from a frequentist confidence interval?

A. A credible interval is always narrower

B. A credible interval assigns posterior probability to the parameter interval conditional on the model and data

C. A confidence interval uses no data

D. They have identical interpretations in all contexts

Correct Answer: B. A credible interval assigns posterior probability to the parameter interval conditional on the model and data

Question 9

What is posterior predictive checking used for?

A. To prove that a prior is objective

B. To compare observed data with data simulated from the fitted model

C. To remove all model uncertainty

D. To replace measurement-system analysis

Correct Answer: B. To compare observed data with data simulated from the fitted model

Question 10

Why must observing a machine condition be distinguished from intervening to set that condition?

A. Observation and intervention always have identical probabilities

B. Intervention may alter causal pathways represented in the model

C. Observation cannot be represented probabilistically

D. Intervention is relevant only to descriptive statistics

Correct Answer: B. Intervention may alter causal pathways represented in the model