Probability and Bayesian Reasoning

Introduction

Business decisions are made with incomplete information. A retailer forecasts demand before sales occur, a bank evaluates a borrower before repayment is known, and a hospital interprets a diagnostic result before the patient’s condition is certain. Probability provides a disciplined language for describing uncertainty, while Bayesian reasoning provides a method for updating beliefs when new evidence becomes available.

In business analytics, probability is not merely a calculation technique. It supports risk assessment, quality control, fraud detection, marketing attribution, credit scoring, medical diagnosis, inventory planning and managerial decision-making. This chapter develops the rules and models required to reason about uncertain events and to interpret evidence responsibly.

Learning Objectives

By the end of this chapter, you should be able to:

• define experiments, outcomes, events and probability models;
• apply addition, multiplication and complement rules;
• distinguish mutually exclusive events from independent events;
• use counting methods to calculate probabilities;
• calculate and interpret conditional probability;
• apply Bayes’ theorem to diagnostic and business evidence;
• calculate expected value and use it in risk-sensitive decisions;
• identify common reasoning errors, data limitations and ethical risks.

1. Foundations of Probability

A random experiment is a process with an uncertain result. An outcome is one possible result, and the sample space, denoted by S, is the set of all possible outcomes. An event is a subset of the sample space.

For example, if a customer either renews or does not renew a subscription, S = {renew, not renew}. The event “the customer renews” contains one outcome. In a more complex setting, an experiment might record both customer segment and renewal status.

A probability P(A) measures the likelihood of event A. It satisfies three axioms:

1. 0 ≤ P(A) ≤ 1;
2. P(S) = 1;
3. if A and B cannot occur together, then P(A ∪ B) = P(A) + P(B).

A probability of 0 represents impossibility and a probability of 1 represents certainty. Probabilities may be estimated from historical data, derived from a theoretical model, or elicited from informed judgment. These sources should not be treated as interchangeable: historical frequencies describe past conditions, whereas subjective probabilities represent informed beliefs about uncertain future events.

Complement Rule

The complement of A, written Aᶜ, is the event that A does not occur.

This rule is useful when the probability of an event is easier to calculate indirectly. If the probability that an order is delivered late is 0.08, the probability that it is not late is 1 − 0.08 = 0.92.

Addition Rule

For any two events A and B:

Here, A ∪ B means “A or B or both,” and A ∩ B means “A and B.” The intersection must be subtracted because it is counted twice when P(A) and P(B) are added.

If A and B are mutually exclusive, they cannot occur together, so P(A ∩ B) = 0 and:

For example, if a randomly selected transaction is either classified as a cash sale or a credit sale, and these categories are mutually exclusive, their probabilities can be added directly.

2. Conditional Probability and Independence

A conditional probability measures the probability of one event given that another event has occurred. The conditional probability of A given B is:

Rearranging gives the multiplication rule:

Equivalently:

The order matters conceptually. P(A | B) asks about A among cases where B is known; P(B | A) asks about B among cases where A is known. They are generally not equal.

Worked Example: Customer Response

Suppose 40% of customers open a promotional email. Among those who open it, 25% make a purchase. The probability that a randomly selected customer both opens the email and purchases is:

P(open and purchase) = P(purchase | open)P(open)

= 0.25 × 0.40 = 0.10.

Thus, 10% of all customers are expected to both open the email and purchase. The 25% figure is a conditional conversion rate, not the overall purchase rate.

Independence

Events A and B are independent if knowing that one occurred does not change the probability of the other:

When P(A) and P(B) are positive, equivalent tests are:

Independence is an assumption about the data-generating process, not a conclusion that should be made merely because two events appear unrelated. For example, the purchase of two products may be dependent because of promotion bundles, customer preferences or seasonality.

Mutual exclusivity and independence are different. Mutually exclusive events with positive probabilities cannot be independent, because if A occurs, B is impossible. Confusing these concepts is a common source of analytical error.

3. Counting Methods

Counting methods are useful when outcomes are equally likely or when a probability model assigns equal weight to combinations.

Fundamental Counting Principle

If a process has k stages, with n₁ possibilities at stage 1, n₂ at stage 2, and so on, the total number of possible outcomes is:

A four-character access code using ten digits at each position has 10⁴ = 10,000 possible codes if repetition is allowed.

Permutations

A permutation counts ordered selections. The number of ways to arrange n distinct objects is:

The number of ordered selections of r objects from n is:

Combinations

A combination counts selections where order does not matter:

For example, the number of committees of three selected from eight employees is C(8,3) = 56. If the committee’s roles are assigned, order or role assignment may matter, and a permutation model may be more appropriate.

Hypergeometric Example

A quality inspector selects 3 items without replacement from a batch of 10, of which 2 are defective. The probability of selecting exactly 1 defective item is:

= (2 × 28) ÷ 120 = 56 ÷ 120 ≈ 0.467.

This model applies because the sample is drawn without replacement from a finite population. If each item were independently classified with a constant defect probability, a binomial model would be more appropriate.

4. Bayes’ Theorem

Bayes’ theorem formalizes the updating of a prior belief after observing evidence. For events A and B, where P(B) > 0:

The terms have distinct meanings:

The denominator can be expanded using the law of total probability. If A and Aᶜ partition the sample space:

Therefore:

Diagnostic Example

Suppose 2% of online transactions are fraudulent. A fraud-screening system flags 90% of fraudulent transactions and also flags 5% of legitimate transactions. If a transaction is flagged, what is the probability that it is actually fraudulent?

Let F denote fraud and + denote a flag.

P(F) = 0.02; P(+ | F) = 0.90; P(+ | Fᶜ) = 0.05; P(Fᶜ) = 0.98.

First calculate the overall flag rate:

P(+) = (0.90)(0.02) + (0.05)(0.98)

= 0.018 + 0.049 = 0.067.

Then:

P(F | +) = (0.90 × 0.02) ÷ 0.067

= 0.018 ÷ 0.067 ≈ 0.269.

Thus, only about 26.9% of flagged transactions are fraudulent. The system may be useful for prioritizing review, but a flag is not proof of fraud. The relatively low base rate of fraud produces many false positives.

A frequency interpretation makes this clearer. In 10,000 transactions, approximately 200 are fraudulent, of which 180 are flagged. Of the 9,800 legitimate transactions, 490 are incorrectly flagged. Among 670 flagged transactions, 180 are fraudulent, giving 180 ÷ 670 ≈ 26.9%.

Bayesian Updating and Likelihood Ratios

Bayesian reasoning can be expressed through odds. The posterior odds equal prior odds multiplied by the likelihood ratio:

This form is especially useful when several independent pieces of evidence arrive sequentially. However, evidence must be conditionally independent—not merely individually unrelated—before likelihood ratios can be multiplied without adjustment.

5. Expected Value and Risk Assessment

Expected value is the probability-weighted average of possible outcomes. For a discrete random variable X with possible values xᵢ and probabilities pᵢ:

The probabilities must satisfy pᵢ ≥ 0 and ∑pᵢ = 1. If X represents monetary payoff, E(X) is measured in currency; if X represents units sold, E(X) is measured in units.

Worked Example: Product Launch

A firm is considering a launch. There is a 0.30 probability of high demand, producing a profit of $120,000, and a 0.70 probability of low demand, producing a loss of $30,000. The expected monetary value is:

E(X) = (0.30)($120,000) + (0.70)(−$30,000)

= $36,000 − $21,000 = $15,000.

The expected value is positive, but this does not mean that the firm will earn $15,000. It is a long-run average across comparable decisions or repeated uncertainty. A manager must also consider cash-flow constraints, downside exposure, risk tolerance and the possibility that the probabilities are poorly estimated.

Variability and Risk

Two decisions may have the same expected value but different risk. Variance measures dispersion around the expected value:

where μ = E(X). The standard deviation is:

Variance is expressed in squared units, while standard deviation has the same units as X. These measures describe uncertainty but do not capture every aspect of risk, such as extreme losses, skewness or liquidity pressure.

For the product launch, μ = $15,000. The variance is:

(120,000 − 15,000)²(0.30) + (−30,000 − 15,000)²(0.70)

= (105,000²)(0.30) + (−45,000²)(0.70)

= 3,307,500,000 + 1,417,500,000

= 4,725,000,000.

The standard deviation is approximately $68,739. The large standard deviation indicates substantial uncertainty relative to the expected profit.

Expected Utility

Expected monetary value assumes that managers value money linearly. In practice, risk aversion may make a sure moderate gain preferable to a gamble with the same expected value. Expected utility represents this by applying a utility function u(x):

The choice of utility function should be justified and may differ across individuals or organizations. Expected value is therefore a useful benchmark, not an automatic decision rule.

6. Decision Trees and the Value of Information

A decision tree represents choices, uncertain events and resulting payoffs. Decision nodes are controlled by the organization; chance nodes represent uncertain outcomes. At a chance node, the expected value is found by multiplying each payoff by its probability and adding the results. The tree is then evaluated from right to left, selecting the best available action at each decision node.

Information has economic value when it changes decisions sufficiently to improve expected outcomes. The expected value of perfect information is:

Perfect information is usually unattainable, but the concept establishes an upper bound on what research, testing or analytics could be worth. The value of sample information is lower because a forecast or test is imperfect and may itself incur cost.

7. Applied Case: Fraud Screening in Digital Payments

This is an illustrative case.

Background

A digital payment company processes 50,000 transactions per day. Management is considering an automated fraud-screening system. Historical data suggest a 1.5% fraud rate. The proposed system detects 92% of fraudulent transactions and incorrectly flags 4% of legitimate transactions.

Problem

Management must determine whether a flagged transaction should be automatically declined, manually reviewed or allowed to proceed. A false positive may frustrate a genuine customer, while a false negative may create financial loss and regulatory exposure.

Analysis

Expected daily fraudulent transactions are 50,000 × 0.015 = 750. The system flags approximately 0.92 × 750 = 690 fraudulent transactions. Legitimate transactions number 49,250, of which 0.04 × 49,250 = 1,970 are falsely flagged. Thus, total flags are approximately 2,660, and the probability that a flagged transaction is fraudulent is 690 ÷ 2,660 ≈ 25.9%.

The posterior probability is much lower than the system’s 92% detection rate because detection rate is P(flag | fraud), whereas management needs P(fraud | flag). These are different conditional probabilities.

Approach and Solution

A sensible policy is risk-based triage:

• high-risk flags receive immediate manual review;
• medium-risk cases may receive additional authentication;
• low-risk cases may be released with monitoring;
• automatic rejection is reserved for cases where the expected cost of fraud clearly exceeds the cost of rejecting a genuine transaction.

The company should estimate the costs of fraud, review, customer inconvenience and regulatory remediation. It should also validate the model on a holdout sample and monitor performance by customer segment, geography and payment method.

Outcome and Learning Points

The principal lesson is that model sensitivity does not equal the probability that a positive signal is correct. Base rates matter. The firm should publish operational definitions, retain an audit trail, monitor false-positive rates and provide an appeal process. A mathematically accurate model can still create unfair outcomes if training data reflect historical bias or if thresholds are applied without considering the consequences for affected customers.

8. Common Errors and Limitations

Base-rate neglect occurs when people focus on a test’s apparent accuracy while ignoring the prevalence of the underlying event. Bayes’ theorem corrects this error.

Gambler’s fallacy is the belief that an independent event becomes more likely because it has not occurred recently. A fair coin remains equally likely to produce heads on each toss.

Prosecutor’s fallacy confuses P(evidence | innocence) with P(innocence | evidence). The same logical error appears in fraud, credit and medical analytics.

Confusing correlation with independence is also dangerous. Statistical association may arise from a common cause, selection bias or time trends. Independence should be supported by a credible process model or tested against appropriate data.

Probability calculations are only as reliable as their assumptions. Historical probabilities may change because of competition, regulation, inflation, technology or customer behavior. Small samples produce unstable estimates, and selection bias can make observed frequencies unrepresentative. Analysts should report uncertainty, sensitivity analyses and the origin of probability estimates.

9. Governance, Ethics and Technology

Bayesian models and machine-learning systems are increasingly used for credit decisions, fraud detection, demand forecasting and customer targeting. Their use requires more than predictive accuracy. Organizations should consider explainability, privacy, fairness, security, accountability and the right to human review.

A probability score should not be presented as a fact. A customer assigned a 0.70 probability of default is not certain to default; the score describes risk under a model and population. Decision-makers should distinguish model output from verified evidence and should avoid using sensitive attributes or proxies in ways that create unlawful discrimination.

Best practice includes documenting data sources, priors, likelihood assumptions, threshold choices and performance metrics; testing calibration as well as discrimination; monitoring population drift; validating results independently; and maintaining governance over model changes. Bayesian updating can improve decisions when new evidence is relevant, reliable and properly incorporated, but it cannot eliminate uncertainty.

Chapter Summary

Probability provides axioms and rules for reasoning about uncertain events. Complements simplify “not” events, the addition rule handles alternatives, and the multiplication rule handles joint events. Conditional probability describes uncertainty after information is known, while independence means that information about one event does not alter the probability of another.

Counting methods support probability calculations for ordered and unordered selections. Bayes’ theorem updates a prior probability using new evidence and is particularly important when base rates are low. Diagnostic accuracy measures such as sensitivity must not be confused with the posterior probability that a positive result is correct. Expected value summarizes probability-weighted outcomes, while variance and standard deviation describe dispersion. Decision trees, expected utility and the value of information extend these ideas to managerial choices.

Effective business use of probability requires clear assumptions, reliable data, sensitivity analysis and ethical governance. Probability supports judgment; it does not replace judgment.

Key Terms

Probability: A numerical measure between 0 and 1 describing the likelihood of an event.

Random experiment: A process whose outcome is uncertain before it occurs.

Outcome: A possible result of a random experiment.

Sample space: The set of all possible outcomes.

Event: A subset of the sample space.

Complement: The event that an event does not occur.

Mutually exclusive events: Events that cannot occur together.

Conditional probability: The probability of one event given that another event is known to have occurred.

Independence: A condition in which knowledge of one event does not change the probability of another.

Permutation: An ordered selection of objects.

Combination: An unordered selection of objects.

Prior probability: Belief about an event before new evidence is observed.

Likelihood: The probability of observed evidence under a specified condition.

Posterior probability: Updated probability after evidence is incorporated.

Bayes’ theorem: A rule for updating probabilities using conditional evidence.

Base rate: The prevalence or prior probability of an event in a population.

Expected value: The probability-weighted average of possible outcomes.

Variance: The probability-weighted squared dispersion around the expected value.

Expected utility: The probability-weighted value of outcomes after applying a utility function.

Expected value of information: The improvement in expected decision value attributable to information.

Review Questions

Short Answer Questions

1. What is the difference between a sample space and an event?
2. State the complement rule and explain when it is useful.
3. Distinguish mutually exclusive events from independent events.
4. What is the difference between a prior probability and a posterior probability?
5. Why can a highly sensitive fraud test still have a low probability that a flagged case is truly fraudulent?

Long Answer Questions

1. Explain the addition and multiplication rules of probability, including their assumptions and business applications.
2. Describe Bayes’ theorem using the concepts of prior, likelihood, evidence and posterior probability.
3. Compare permutations and combinations and provide an example of when each is appropriate.
4. Explain expected value, variance and standard deviation as tools for managerial risk assessment.
5. Discuss the ethical and governance requirements for using probability-based automated decisions.

Analytical / Application Questions

1. A retailer estimates that 8% of customers are high-value. A marketing model identifies 75% of high-value customers and incorrectly identifies 12% of other customers. Calculate the probability that a customer identified by the model is actually high-value.
2. A project has a 0.40 probability of generating a $90,000 profit and a 0.60 probability of producing a $20,000 loss. Calculate its expected monetary value and explain whether expected value alone is sufficient for the decision.
3. A quality-control test is applied to a process with a 3% defect rate. It detects 95% of defective units and falsely flags 6% of good units. Explain how Bayes’ theorem should be used to evaluate the test and why the base rate matters.

Multiple Choice Questions with Answers

Question 1

If P(A) = 0.30 and P(B) = 0.20, and A and B are mutually exclusive, what is P(A ∪ B)?

A. 0.06

B. 0.10

C. 0.50

D. 1.50

Correct Answer: C. 0.50

Question 2

If P(A) = 0.40, what is P(Aᶜ)?

A. 0.20

B. 0.40

C. 0.60

D. 1.40

Correct Answer: C. 0.60

Question 3

Which expression defines conditional probability P(A | B), assuming P(B) > 0?

A. P(A) ÷ P(B)

B. P(A ∩ B) ÷ P(B)

C. P(A ∪ B) ÷ P(B)

D. P(B) ÷ P(A ∩ B)

Correct Answer: B. P(A ∩ B) ÷ P(B)

Question 4

If A and B are independent, which relationship must hold?

A. P(A ∩ B) = P(A) + P(B)

B. P(A | B) = 0

C. P(A ∩ B) = P(A)P(B)

D. P(A) = 1 − P(B)

Correct Answer: C. P(A ∩ B) = P(A)P(B)

Question 5

How many committees of 2 can be selected from 6 employees when order does not matter?

A. 6

B. 12

C. 15

D. 36

Correct Answer: C. 15

Question 6

A business outcome is $10,000 with probability 0.7 and −$4,000 with probability 0.3. What is its expected value?

A. $2,200

B. $5,800

C. $7,000

D. $10,600

Correct Answer: A. $2,200

Question 7

In Bayes’ theorem, P(A) usually represents which quantity?

A. The posterior probability

B. The prior probability

C. The false-positive rate

D. The sample size

Correct Answer: B. The prior probability

Question 8

A disease affects 1% of a population. A test detects 90% of diseased cases and falsely flags 5% of healthy cases. Approximately what is the probability that a flagged person has the disease?

A. 1.0%

B. 5.0%

C. 15.4%

D. 90.0%

Correct Answer: C. 15.4%

Question 9

Which statement about expected value is correct?

A. It guarantees the outcome of one decision.

B. It is a probability-weighted average of possible outcomes.

C. It always identifies the least risky option.

D. It can be calculated without probabilities.

Correct Answer: B. It is a probability-weighted average of possible outcomes.

Question 10

Why is a positive predictive value often lower than a test’s sensitivity when the underlying event is rare?

A. Rare events have no conditional probabilities.

B. The base rate contributes many non-event cases that may be false positives.

C. Sensitivity measures the number of legitimate cases.

D. Bayes’ theorem ignores the prior probability.

Correct Answer: B. The base rate contributes many non-event cases that may be false positives.