Uncertainty, Probabilistic Reasoning and Decision Theory

Introduction and Context

Artificial intelligence systems rarely operate with complete, reliable, and unambiguous information. A medical diagnosis may be based on incomplete symptoms, a robot may receive noisy sensor readings, and a fraud-detection system may have to act before a transaction is fully investigated. In such settings, intelligent behaviour requires more than pattern recognition: it requires explicit representation of uncertainty, rational updating of beliefs, and selection of actions whose expected consequences are acceptable.

This chapter examines major formalisms for reasoning and decision-making under uncertainty. Probability provides a mathematically coherent language for uncertain events. Bayesian inference describes how beliefs should change when evidence is observed. Bayesian networks represent dependencies among variables, while Markov models represent evolving states over time. Fuzzy logic addresses graded concepts and approximate linguistic reasoning. Utility theory distinguishes desirable outcomes from merely probable outcomes. Sequential decision models, including Markov decision processes and partially observable models, extend these ideas to repeated action under uncertainty.

The central distinction is between uncertainty about the state of the world and preferences concerning possible outcomes. Probability models beliefs; utility models value; decision theory combines them. A technically sophisticated AI system must also account for data quality, model assumptions, computational cost, fairness, explainability, and the consequences of incorrect decisions.

Learning Objectives

After completing this chapter, the learner should be able to:

1. Explain aleatory and epistemic uncertainty and represent uncertain events using probability.

2. Apply conditional probability, Bayes’ theorem, independence, and expected utility.

3. Construct and interpret Bayesian networks for diagnostic and predictive reasoning.

4. Explain Markov chains, hidden Markov models, and their principal inference tasks.

5. Distinguish probabilistic uncertainty from vagueness and use fuzzy logic appropriately.

6. Formulate sequential decisions using Markov decision processes and partially observable models.

7. Evaluate assumptions, limitations, ethical risks, and governance requirements in uncertain AI systems.

1. The Nature of Uncertainty in Artificial Intelligence

Uncertainty occurs when an agent cannot determine the true state of a system with certainty. Two broad sources are useful:

• Aleatory uncertainty is inherent variability in a process. For example, even under identical apparent conditions, a component may fail at different times.
• Epistemic uncertainty results from limited knowledge, insufficient data, measurement error, or model inadequacy. It may be reduced by collecting better evidence, although not always eliminated.

A sensor reading of 38.2°C may be uncertain because the sensor is noisy (measurement uncertainty), because the patient’s true temperature is not known (state uncertainty), or because the relationship between temperature and disease is imperfect (model uncertainty).

Uncertainty should not be confused with vagueness. The statement “the patient has a high fever” may be vague because the boundary between high and not high is not precisely defined. The statement “there is a 0.7 probability that the patient has influenza” is probabilistic: it expresses uncertainty about a comparatively well-defined proposition. Fuzzy logic is often useful for the first situation; probability is designed for the second.

1.1 Probability as a Representation of Belief

Let Ω be a sample space containing possible outcomes and let A be an event, which is a subset of Ω. A probability measure P assigns a value between 0 and 1 to events and satisfies:

P(Ω) = 1,

P(A) ≥ 0,

and, for mutually exclusive events A1, A2, ..., P(∪i Ai) = Σi P(Ai).

The last property is countable additivity. These axioms do not specify how probabilities are obtained. They provide consistency conditions for reasoning once probabilities have been assigned from data, expert judgement, or a model.

A probability of 0.8 does not mean that an event will occur in exactly 80% of a small set of trials. It may represent a long-run frequency, a calibrated forecast, or an agent’s degree of belief. The interpretation must be appropriate to the application.

For events A and B with P(B) > 0, conditional probability is:

P(A | B) = P(A ∩ B) / P(B).

The multiplication rule follows directly:

P(A ∩ B) = P(A | B)P(B) = P(B | A)P(A).

Two events are independent if P(A ∩ B) = P(A)P(B). Equivalently, when the relevant probabilities are defined, P(A | B) = P(A). Independence is a substantive modelling assumption, not a consequence of the fact that two variables have different names. Incorrect independence assumptions can produce seriously misleading conclusions.

2. Bayesian Reasoning

Bayes’ theorem is obtained by equating the two forms of the multiplication rule:

P(A | B) = [P(B | A)P(A)] / P(B).

Here, P(A) is the prior probability, P(B | A) is the likelihood of the evidence under hypothesis A, P(B) is the evidence probability, and P(A | B) is the posterior probability. For mutually exclusive and exhaustive hypotheses H1, ..., Hn:

P(Hi | E) = [P(E | Hi)P(Hi)] / [Σj P(E | Hj)P(Hj)].

Bayesian reasoning is especially important when a test or observation is informative but imperfect. A common professional error is to interpret P(E | H) as P(H | E). These quantities are generally different.

Worked Example: Medical Screening

Suppose a disease has prevalence P(D) = 0.01. A test has sensitivity P(+ | D) = 0.90 and specificity 0.95, so its false-positive rate is P(+ | not D) = 0.05. The probability of a positive result is:

P(+) = P(+ | D)P(D) + P(+ | not D)P(not D)

= (0.90)(0.01) + (0.05)(0.99)

= 0.009 + 0.0495 = 0.0585.

Therefore:

P(D | +) = (0.90 × 0.01) / 0.0585

= 0.009 / 0.0585 ≈ 0.1538.

Thus, a positive result implies approximately a 15.4% probability of disease, not 90%. The low prevalence means that false positives form a substantial proportion of all positive results. In practice, confirmatory testing, patient characteristics, test calibration, and consequences of treatment must also be considered.

Odds and Likelihood Ratios

Bayesian updating can also be expressed in odds form:

Posterior odds = Prior odds × Likelihood ratio.

For a positive test, the positive likelihood ratio is:

LR+ = P(+ | D) / P(+ | not D).

In the example, LR+ = 0.90 / 0.05 = 18. The prior odds are 0.01/0.99. Multiplying by 18 gives posterior odds approximately 0.1818, which converts to probability 0.1818/(1 + 0.1818) ≈ 0.1538. Odds form is useful when several conditionally independent pieces of evidence are incorporated sequentially, although conditional independence must be justified.

Bayesian Updating and Model Calibration

A probabilistic prediction is calibrated if, among cases assigned probability p, the event occurs approximately p proportion of the time. Calibration differs from discrimination. A model may rank positive cases above negative cases effectively while producing probabilities that are systematically too high or too low. Reliability diagrams, the Brier score, log loss, and calibration curves are useful evaluation tools.

The Brier score for binary outcomes is commonly defined as:

BS = (1/N)Σi(pi − yi)^2,

where pi is the predicted probability and yi is 0 or 1. Lower values are better. Log loss is particularly sensitive to confident incorrect predictions and is often appropriate when probability quality matters.

3. Bayesian Networks

A Bayesian network is a directed acyclic graph (DAG) whose nodes represent random variables and whose directed edges represent conditional dependencies. Each node has a conditional probability distribution given its parent nodes. The joint distribution factorizes as:

P(X1, X2, ..., Xn) = Πi P(Xi | Parents(Xi)).

This factorization can reduce the number of parameters required compared with representing every joint probability directly.

Example Structure

Consider a network with variables Rain, Sprinkler, WetGrass, and Traffic. A possible structure is:

Rain → WetGrass,

Sprinkler → WetGrass,

Rain → Traffic.

The joint distribution is:

P(Rain, Sprinkler, WetGrass, Traffic)

= P(Rain)P(Sprinkler)P(WetGrass | Rain, Sprinkler)P(Traffic | Rain).

The graph encodes conditional independencies. In this example, WetGrass depends directly on Rain and Sprinkler. Traffic is conditionally independent of Sprinkler given Rain, under the model. Such independencies must be examined using graph-separation principles and domain knowledge rather than assumed automatically.

Inference in Bayesian Networks

Typical queries include:

• Predictive inference: estimating effects from causes, such as P(WetGrass | Rain).
• Diagnostic inference: reasoning from effects to causes, such as P(Rain | WetGrass).
• Intercausal reasoning: assessing one cause after observing another cause and a common effect.
• Most probable explanation: identifying the most likely assignment of hidden variables.

Exact inference may use enumeration, variable elimination, or junction-tree methods. Inference can become computationally difficult in densely connected networks. Approximate approaches include rejection sampling, likelihood weighting, Markov chain Monte Carlo, and variational methods.

Learning Bayesian Networks

Structure learning attempts to infer graph connections from data, while parameter learning estimates conditional probabilities. With complete data and a known structure, maximum likelihood estimation is straightforward for discrete tables. With missing data or hidden variables, expectation-maximization is often used. Structure learning may rely on constraint-based tests, score-based search, or hybrid methods.

Causal interpretation requires more than predictive association. A Bayesian network can be used causally only when its graph, variables, and interventions are justified by an appropriate causal model. Observing that two variables are associated does not establish that changing one will change the other.

4. Markov Models for Temporal Reasoning

Static Bayesian networks describe relationships among variables at a particular representation of a problem. Markov models describe how systems evolve over time.

4.1 Markov Chains

A discrete-time Markov chain has states S and transition probabilities:

P(Xt+1 = j | Xt = i, Xt−1, ..., X0) = P(Xt+1 = j | Xt = i) = Tij.

This is the first-order Markov property: the next state depends on the current state, not on the entire past, once the current state is known. The transition matrix T has nonnegative entries and each row sums to 1.

If πt is a row vector containing state probabilities at time t, then:

πt+1 = πt T.

After k transitions:

πt+k = πt T^k.

A stationary distribution π satisfies:

π = πT,

Σi πi = 1,

πi ≥ 0.

A unique limiting stationary distribution is not guaranteed. Conditions such as irreducibility and aperiodicity are commonly required for convergence in finite chains.

Worked Example: Machine Condition

A machine is either Healthy (H) or Degraded (D). Suppose the transition matrix is:

T = [[0.9, 0.1], [0.4, 0.6]],

where rows represent the current state H or D and columns represent the next state H or D. If the machine is initially healthy, π0 = [1, 0]. After one period:

π1 = [1, 0]T = [0.9, 0.1].

After two periods:

π2 = [0.9, 0.1]T

= [0.9(0.9) + 0.1(0.4), 0.9(0.1) + 0.1(0.6)]

= [0.85, 0.15].

The probability of degradation after two periods is therefore 0.15.

4.2 Hidden Markov Models

In a hidden Markov model (HMM), the state Xt is not directly observed. Instead, an observation Yt is generated according to an emission distribution P(Yt | Xt). The model comprises:

• initial-state probabilities P(X1),
• transition probabilities P(Xt | Xt−1),
• emission probabilities P(Yt | Xt).

The principal tasks are:

1. Evaluation: calculate P(Y1:T), often using the forward algorithm.

2. Decoding: find the most likely state sequence, commonly using the Viterbi algorithm.

3. Learning: estimate parameters, often through Baum–Welch, an expectation-maximization procedure.

HMMs have been used in speech recognition, activity recognition, biological sequence analysis, and fault diagnosis. Their assumptions can be restrictive: the state must summarize all relevant history, transitions are often stationary, and observations are conditionally independent given the current state.

5. Fuzzy Logic and Approximate Reasoning

Fuzzy logic represents degrees of membership in concepts rather than probabilities of events. A fuzzy set A on universe X is characterized by a membership function μA(x) in [0,1]. A value μA(x) = 0.8 means that x belongs to the concept A to degree 0.8; it does not mean that there is an 80% probability that x belongs to A.

A fuzzy rule may be expressed as:

IF temperature is high AND pressure is low THEN risk is elevated.

A typical fuzzy inference system includes:

1. Fuzzification: convert crisp measurements into membership degrees.

2. Rule evaluation: determine the strength of each linguistic rule.

3. Aggregation: combine the outputs of applicable rules.

4. Defuzzification: convert the aggregated fuzzy output into a crisp action or value.

Common logical operators include the minimum or product for conjunction and the maximum for disjunction, although other t-norms and t-conorms are possible. In a Mamdani system, output fuzzy sets are aggregated and often converted using the centroid method:

z* = [∫ z μ(z) dz] / [∫ μ(z) dz],

provided the denominator is nonzero and the output is represented continuously. For discrete output levels, summations replace integrals.

Fuzzy logic is valuable when expert knowledge is linguistic, boundaries are gradual, and an interpretable rule base is desirable. It is not a replacement for probability when the key issue is uncertainty about whether an event occurred or will occur. Hybrid neuro-fuzzy systems can learn membership functions or rules from data, but they may sacrifice interpretability and require careful validation.

6. Utility Theory and Rational Choice

Probability alone cannot determine what an agent should do. A decision requires preferences over outcomes. Utility theory represents those preferences numerically under specified rationality assumptions.

Let a be an action and S a state of the world. If U(a, S) is the utility of taking action a when state S occurs, the expected utility is:

EU(a) = Σs P(s | evidence)U(a, s),

for discrete states. For continuous outcomes, the sum is replaced by an integral. A rational expected-utility decision selects an action maximizing EU(a), subject to constraints.

Worked Example: Inspection Decision

A manufacturer can ship a product immediately or inspect it first. There is a 0.20 probability that a defect exists. Shipping a defective product incurs a loss of 100 monetary units; shipping a non-defective product gives utility 20. Inspection costs 3 units and, for simplicity, is assumed to reveal the condition perfectly.

Without inspection:

EU(ship) = (0.20)(−100) + (0.80)(20)

= −20 + 16 = −4.

If inspection is perfect, the manufacturer ships only when the product is non-defective. The expected utility before paying for inspection is:

(0.20)(0) + (0.80)(20) = 16.

After inspection cost:

EU(inspect) = 16 − 3 = 13.

Inspection is preferred because 13 > −4. The value of perfect information is 16 − (−4) = 20, and the net value after inspection cost is 17 relative to shipping immediately. This calculation assumes that utilities are commensurable, the probabilities are reliable, and inspection has no other effects.

Risk Attitudes

Expected monetary value assumes linear utility for money and therefore risk neutrality. Real decision-makers may be risk-averse. A concave utility function, such as U(w) = √w for nonnegative wealth w, represents diminishing marginal utility of wealth. Risk attitude should be elicited or estimated rather than silently imposed. In safety-critical systems, worst-case constraints, risk limits, or lexicographic priorities may be more appropriate than unconstrained expected utility maximization.

Multi-attribute decisions require care. Cost, safety, fairness, delay, and environmental impact may be measured in different units. Weighted sums can be useful, but weights must be justified, sensitivity-tested, and reviewed with stakeholders. A high expected utility does not automatically make a decision ethically acceptable if it imposes severe harms on a minority group.

7. Sequential Decision Models

Many AI tasks involve repeated decisions. An action changes the state, produces a reward or cost, and creates a new decision problem. A Markov decision process (MDP) is defined by:

• states S,
• actions A,
• transition model P(s' | s, a),
• reward function R(s, a, s'),
• discount factor γ, where 0 ≤ γ < 1 for infinite-horizon discounted problems.

The Bellman optimality equation for a value function is:

V(s) = maxa Σs' P(s' | s, a)[R(s, a, s') + γV(s')].

The value of a policy π is:

Vπ(s) = Eπ[Σt=0∞ γt Rt | S0 = s].

The discount factor reflects time preference or mathematical emphasis on near-term rewards. It should not be selected solely for numerical convenience when it has substantive ethical implications.

Algorithms include value iteration, policy iteration, and model-free reinforcement learning. Value iteration repeatedly applies the Bellman optimality operator until values converge under suitable conditions. Policy iteration alternates between evaluating a policy and improving it. In model-free methods, an agent estimates values or policies from interaction rather than an explicitly known transition model.

Partially Observable Markov Decision Processes

In a POMDP, the true state is hidden. The agent receives observations and maintains a belief state b(s), a probability distribution over possible states. After taking action a and receiving observation o, belief updating follows:

b'(s') = η P(o | s', a) Σs P(s' | s, a)b(s),

where η is a normalizing constant ensuring that Σs' b'(s') = 1. The belief state acts as a sufficient statistic for the history under the model, converting the POMDP into a continuous-state belief MDP.

POMDPs are useful for robot navigation, dialogue management, maintenance, and clinical planning. Exact solution can be computationally expensive because the belief space is continuous. Practical systems use point-based planning, approximate dynamic programming, simulation, or receding-horizon methods.

Exploration and Exploitation

A sequential learner must decide whether to exploit the best currently known action or explore alternatives that may yield better information. This is the exploration–exploitation dilemma. Bandit models isolate this issue when actions generate rewards but do not change a broader state. Strategies include epsilon-greedy selection, upper-confidence methods, and Bayesian sampling. Exploration in real systems must be bounded by safety constraints; an algorithm should not experiment freely when mistakes can harm people.

8. Integrated Case Study: Predictive Maintenance for a Data-Centre Cooling System

The following is an illustrative case study rather than a report of a particular company.

Background

A data centre operates cooling units whose failure can cause service disruption and equipment damage. Sensors measure temperature, vibration, airflow, and power consumption. Measurements are noisy, and some failures are rare. The operator must decide whether to continue operation, inspect a unit, reduce its load, or schedule maintenance.

Problem

A simple threshold rule generates many false alarms because a temporary temperature increase is not necessarily a failure. Conversely, waiting for a definitive fault signal may leave insufficient time for safe intervention. The system therefore needs to combine uncertain observations, temporal patterns, action costs, and service-level requirements.

Analysis

A Bayesian network can represent static relationships among operating load, ambient temperature, airflow restriction, vibration, and failure risk. A hidden Markov model can represent latent equipment conditions such as Healthy, Degrading, and Critical, with sensor readings emitted probabilistically from each state. The model can estimate the probability that the unit is degrading even when no single sensor is decisive.

A decision layer assigns utilities to actions. Continuing operation has high short-term utility but a large negative utility if failure occurs. Inspection has a direct cost and may reduce uncertainty. Preventive maintenance has a larger immediate cost but can avoid catastrophic downtime. If inspections and interventions affect future condition, the problem is sequential and is better represented by an MDP or POMDP.

Approach and Solution

The proposed architecture has five stages:

1. Validate and time-align sensor data, quantify missingness, and detect sensor drift.

2. Estimate a temporal state model using historical maintenance records and expert constraints.

3. Calibrate predicted probabilities on a temporally separated validation set.

4. Compute action values using downtime cost, maintenance cost, safety limits, and operational constraints.

5. Require human approval for high-impact interventions and log model inputs, predictions, actions, and outcomes.

Suppose the current posterior probability of a critical state is 0.15. Continuing operation has utility 8 if the unit is not critical and −80 if it is critical. Immediate maintenance has utility −10 regardless of state. Ignoring other future effects:

EU(continue) = (0.85)(8) + (0.15)(−80)

= 6.8 − 12 = −5.2.

EU(maintenance) = −10.

Under these simplified assumptions, maintenance is not preferred because −5.2 is greater than −10. However, if maintenance reduces the probability of subsequent catastrophic failure, a sequential model may make maintenance preferable. This illustrates why one-step calculations can be inadequate for operational planning.

Outcome

A successful deployment would not be judged solely by classification accuracy. Relevant measures include calibration, missed-failure rate, false-alarm burden, avoided downtime, maintenance cost, response time, and equity across equipment types. The system should be monitored after deployment because operating conditions, sensor behaviour, and failure mechanisms may change.

Learning Points

The case demonstrates that prediction and decision are different tasks. A probability estimate must be connected to consequences and feasible actions. Temporal dependence matters, perfect observability is unrealistic, and a model that performs well retrospectively may fail when maintenance policies change the data-generating process. Human oversight and auditability are essential because the cost of a false negative may be much greater than the cost of a false positive.

9. Limitations, Governance, and Professional Practice

Data and Model Risk

Probability estimates inherit bias and error from data. Rare-event datasets may contain few positive examples, while historical decisions may reflect past discrimination or inconsistent recording. Distribution shift occurs when the deployment population, sensors, policies, or environment differ from the training data. Uncertainty estimates should therefore include out-of-distribution detection, stress testing, and sensitivity analysis where feasible.

A model can be mathematically coherent yet operationally wrong because variables were defined incorrectly, labels were affected by intervention, or missingness was informative. Data leakage can create unrealistically strong validation results. Temporal and group-based splits are often more appropriate than random splitting when deployment involves future cases or distinct populations.

Fairness and Accountability

Decisions based on risk scores can distribute benefits and burdens unequally. Calibration, equalized error rates, predictive parity, and individual fairness are different criteria and cannot generally all be satisfied simultaneously when base rates differ. The appropriate criterion depends on the application, legal context, and ethical purpose. Fairness analysis should examine the entire decision process, including access to testing, human overrides, and downstream consequences.

High-impact systems should document the decision purpose, data provenance, model assumptions, uncertainty, performance by relevant subgroups, escalation procedures, and appeal mechanisms. Privacy-preserving data management, access controls, retention limits, and compliance with applicable data-protection and sectoral regulations are required. Probabilistic outputs should not be presented as objective facts when they depend on contestable assumptions.

Best Practices

1. Define the decision, not merely the prediction target.

2. Separate aleatory variability from lack of knowledge where possible.

3. Make conditional-independence and stationarity assumptions explicit.

4. Calibrate probabilities and report uncertainty, not just accuracy.

5. Use utility and cost analysis with stakeholder-reviewed values.

6. Test sensitivity to priors, model structure, reward weights, and discount factors.

7. Evaluate temporal drift, subgroup performance, and rare but consequential failures.

8. Keep humans meaningfully involved in high-impact or irreversible decisions.

9. Log evidence, model version, output, action, and outcome for auditability.

10. Establish monitoring, retraining, rollback, and incident-response procedures.

10. Contemporary Developments and Research Directions

Modern AI increasingly combines probabilistic models with deep learning. Deep generative models can represent complex distributions, while Bayesian deep learning and ensemble methods attempt to quantify predictive uncertainty. Calibration remains difficult, especially under distribution shift and for large language models. Conformal prediction provides distribution-free prediction sets under exchangeability assumptions, offering a way to control marginal coverage, although coverage may deteriorate when assumptions fail and does not by itself solve decision or fairness problems.

Causal probabilistic models are being integrated with machine learning to distinguish intervention effects from observational associations. Neural state-space models, differentiable filters, and learned simulators support temporal inference and planning. Safe reinforcement learning adds constraints, risk-sensitive objectives, and uncertainty-aware exploration. Distributionally robust optimization seeks policies that perform acceptably across a set of plausible data-generating distributions rather than optimizing only for an estimated nominal distribution.

These developments do not remove the need for domain knowledge. More expressive models can be harder to verify, calibrate, explain, and govern. In postgraduate practice, technical innovation should therefore be evaluated together with assumptions, evidence, computational requirements, failure modes, and institutional accountability.

Chapter Summary

Uncertainty is unavoidable in intelligent systems and may arise from inherent variability, incomplete knowledge, noisy observations, or inadequate models. Probability provides rules for coherent belief representation, while Bayes’ theorem updates beliefs when evidence is observed. Bayesian networks compactly represent conditional dependencies and support diagnostic, predictive, and explanatory inference. Markov chains and hidden Markov models represent temporal evolution when the current state summarizes relevant history.

Fuzzy logic addresses vagueness and graded concepts, which must be distinguished from probabilistic uncertainty. Utility theory represents preferences over consequences, and expected utility combines beliefs with values to select actions. Markov decision processes extend decision theory to repeated action, while POMDPs maintain belief distributions when system states are hidden. Practical AI requires calibration, sensitivity analysis, monitoring, fairness assessment, privacy protection, and human accountability. The most important professional principle is to design for the decision context: a prediction is useful only when its uncertainty, consequences, and permissible actions are properly understood.

Key Terms

Probability: A numerical representation of uncertainty about events.

Conditional probability: The probability of an event given that another event is known.

Bayes’ theorem: A rule for updating a prior belief using observed evidence.

Prior: Belief assigned to a hypothesis before considering current evidence.

Likelihood: Probability of observed evidence under a specified hypothesis.

Posterior: Updated probability of a hypothesis after evidence is considered.

Calibration: Agreement between predicted probabilities and observed frequencies.

Bayesian network: A directed acyclic graph with conditional probability distributions.

Conditional independence: Independence between variables given another variable or set of variables.

Markov property: The assumption that the next state depends on the current state rather than the full history.

Markov chain: A stochastic process described by states and transition probabilities.

Hidden Markov model: A temporal model with unobserved states generating observations.

Fuzzy set: A set whose elements have degrees of membership between zero and one.

Utility: A numerical representation of preference or value assigned to an outcome.

Expected utility: Probability-weighted average utility of an action’s possible outcomes.

Markov decision process: A sequential decision model with states, actions, transitions, rewards, and usually discounting.

POMDP: A partially observable Markov decision process in which the agent maintains beliefs about hidden states.

Policy: A rule specifying an action for each state or belief state.

Exploration–exploitation trade-off: The tension between gathering information and using the best current option.

Distribution shift: A change between the data-generating conditions used for training and deployment.

Review Questions

Short Answer Questions

1. Distinguish aleatory uncertainty from epistemic uncertainty.

2. State Bayes’ theorem and identify its prior, likelihood, evidence, and posterior terms.

3. What conditional independence assumption is represented by a Bayesian network edge structure?

4. Explain the difference between fuzzy membership and probability.

5. What is the Markov property?

Long Answer Questions

1. Explain Bayesian networks, their factorization of joint distributions, common inference tasks, and computational limitations.

2. Compare Markov chains and hidden Markov models, including their assumptions and major algorithms.

3. Explain expected utility theory and discuss how risk attitude affects decisions.

4. Describe the formulation and solution concepts of a Markov decision process.

5. Critically discuss calibration, distribution shift, fairness, and accountability in probabilistic AI systems.

Analytical / Application Questions

1. A disease has prevalence 0.02. A test has sensitivity 0.95 and specificity 0.90. Calculate the probability of disease given a positive test and interpret the result.

2. A system may continue operation, inspect equipment, or shut it down. Design a utility table and explain what additional probabilities and future effects are needed to choose rationally.

3. Propose a Bayesian or POMDP-based architecture for a robot navigating with noisy sensors. State the variables, assumptions, belief update process, actions, rewards, and safety constraints.

Multiple Choice Questions with Answers

Question 1

A screening test has P(D)=0.01, P(+|D)=0.90, and P(+|not D)=0.05. What is P(D|+), approximately?

A. 0.0090

B. 0.0500

C. 0.1538

D. 0.9000

Correct Answer: C. 0.1538

Question 2

Which expression correctly represents the factorization of a Bayesian network?

A. P(X1,...,Xn)=Σi P(Xi)

B. P(X1,...,Xn)=Πi P(Xi | Parents(Xi))

C. P(X1,...,Xn)=Πi P(Parents(Xi) | Xi)

D. P(X1,...,Xn)=P(X1) + P(Xn)

Correct Answer: B. P(X1,...,Xn)=Πi P(Xi | Parents(Xi))

Question 3

For a Markov chain with row-vector state distribution πt and transition matrix T, the next distribution is:

A. πt+1 = Tπt

B. πt+1 = πt/T

C. πt+1 = πtT

D. πt+1 = πt + T

Correct Answer: C. πt+1 = πtT

Question 4

What does a membership value of 0.8 in a fuzzy set mean?

A. The event has probability 0.8

B. The element belongs to the concept to degree 0.8

C. The event will occur in 80% of trials

D. The hypothesis has posterior odds of 0.8

Correct Answer: B. The element belongs to the concept to degree 0.8

Question 5

Which quantity combines probabilities of states with utilities of consequences?

A. Expected utility

B. Specificity

C. Transition entropy

D. Membership degree

Correct Answer: A. Expected utility

Question 6

In a hidden Markov model, which component describes the probability of an observation given the hidden state?

A. Initial distribution

B. Transition distribution

C. Emission distribution

D. Stationary distribution

Correct Answer: C. Emission distribution

Question 7

Which condition is commonly associated with convergence of a finite Markov chain to a unique stationary distribution?

A. Irreducibility and aperiodicity

B. Zero transition probabilities everywhere

C. Exactly two states only

D. Deterministic rewards

Correct Answer: A. Irreducibility and aperiodicity

Question 8

In a POMDP, what does the belief state represent?

A. A deterministic action sequence

B. A probability distribution over possible hidden states

C. The model’s discount factor

D. The complete transition matrix only

Correct Answer: B. A probability distribution over possible hidden states

Question 9

In an MDP with discount factor γ, which expression is the Bellman optimality equation?

A. V*(s)=mina R(s,a)

B. V(s)=maxa Σs'P(s'|s,a)[R(s,a,s')+γV(s')]

C. V*(s)=Σa P(a|s)

D. V*(s)=γ/R(s,a,s')

Correct Answer: B. V(s)=maxa Σs'P(s'|s,a)[R(s,a,s')+γV(s')]

Question 10

A calibrated group of predictions assigned probability 0.7 should experience the predicted event at approximately what frequency?

A. 0.07

B. 0.30

C. 0.70

D. 1.00

Correct Answer: C. 0.70