Probability Distributions and Sampling Distributions

Introduction and Context

Probability distributions provide a systematic way to describe uncertainty. In business analytics, uncertainty appears in customer arrivals, product defects, transaction values, delivery times, claims, defaults and many other processes. A probability distribution connects possible outcomes with their probabilities and therefore supports forecasting, risk assessment, quality control and statistical inference.

This chapter develops four closely related ideas. First, probability distributions describe random variables. Second, common discrete distributions—especially the binomial and Poisson distributions—model counts. Third, the normal distribution models continuous measurements and provides an important approximation to several discrete distributions. Finally, sampling distributions explain how statistics such as sample means and sample proportions vary from sample to sample. The law of large numbers and the central limit theorem explain why sample-based conclusions become more reliable as sample size increases.

Learning Objectives

By the end of this chapter, you should be able to:

1. Distinguish discrete and continuous random variables and interpret probability distributions.
2. Apply the binomial, Poisson and normal distributions under appropriate assumptions.
3. Explain the law of large numbers and the central limit theorem.
4. Calculate and interpret sampling distributions, standard errors and z-scores.
5. Select an appropriate distribution for a practical analytics problem and recognise its limitations.
6. Communicate probabilistic results without overstating their certainty.

1. Random Variables and Probability Distributions

A random variable assigns a numerical value to the outcome of a random process. A discrete random variable has countable outcomes, such as the number of complaints received in a day. A continuous random variable can take any value within an interval, such as delivery time or customer spending.

A probability mass function (PMF) gives the probability of each value of a discrete random variable X:

A valid PMF must satisfy:

A probability density function (PDF), written f(x), describes the relative concentration of probability for a continuous variable. For a continuous variable, the probability of one exact point is zero; probabilities are calculated over intervals:

The cumulative distribution function (CDF) applies to both types of random variable:

The expected value, or mean, is the probability-weighted long-run average. For a discrete random variable:

The variance measures dispersion around the mean:

The standard deviation is the positive square root of variance:

For continuous variables, summation is replaced by integration. Variance is expressed in squared units, while standard deviation has the same units as X.

Two useful rules are:

and, for constants a and b,

Adding a constant changes the mean but not the variance; multiplying by b changes the standard deviation by |b|.

2. Important Discrete Distributions

2.1 The Bernoulli Distribution

A Bernoulli trial has two possible outcomes, conventionally called success and failure. If the probability of success is p, then the probability of failure is 1 − p. A Bernoulli random variable X has:

Its mean and variance are:

The model assumes one trial, two mutually exclusive outcomes, and a fixed success probability.

2.2 The Binomial Distribution

The binomial distribution counts successes in n Bernoulli trials. It is appropriate when:

• the number of trials n is fixed;
• each trial has two outcomes;
• trials are independent, or dependence is negligible;
• the probability of success p is constant;
• the random variable counts successes.

If X is the number of successes, then:

where C(n,x) = n!/(x!(n−x)!). The mean and variance are:

Worked example. Suppose 8% of online orders are returned. For five independently selected orders, find the probability that exactly one is returned.

Here n = 5, p = 0.08 and x = 1:

Since C(5,1) = 5 and 0.92⁴ ≈ 0.7164,

The probability is approximately 0.287, or 28.7%.

The binomial model should not be used automatically when sampling without replacement. If a sample is a substantial fraction of a finite population, the trials are dependent and a hypergeometric model may be more suitable. When the population is large relative to the sample, the binomial approximation may be reasonable.

2.3 The Poisson Distribution

The Poisson distribution models the number of events occurring in a fixed interval of time, area, distance or volume. It is useful for calls per hour, claims per day, defects per metre or arrivals per minute. If the average rate is λ, then:

Here λ is both the expected count and the variance:

The assumptions are that events occur independently, the rate is stable over the interval, and simultaneous events are sufficiently unlikely. In practice, a changing rate or clustering of events can invalidate the model.

Worked example. A help desk receives an average of 3 calls per 10 minutes. Assuming a Poisson process, the probability of exactly 5 calls in 10 minutes is:

Thus the probability is approximately 10.1%.

If the interval changes, λ must change proportionally. For example, an average of 3 calls per 10 minutes corresponds to an average of 9 calls per 30 minutes, assuming a constant rate.

The Poisson distribution is also commonly used as an approximation to a binomial distribution when n is large, p is small and λ = np is moderate. The approximation should be justified rather than treated as exact.

2.4 The Hypergeometric Distribution

When a sample of size n is drawn without replacement from a finite population of size N containing K successes, the number X of successes follows a hypergeometric distribution:

Unlike binomial trials, probabilities change after each draw because the population composition changes. This model is relevant to auditing a finite batch, selecting files from a fixed folder or inspecting a shipment without replacement.

3. The Normal Distribution

The normal distribution is a continuous, symmetric, bell-shaped distribution determined by its mean μ and standard deviation σ. Its density is:

The parameters must satisfy σ > 0. The mean determines the centre and the standard deviation determines spread. The total area under the curve is 1, and the curve is symmetric about μ.

A normal variable can be standardised using the z-score:

The resulting standard normal variable has mean 0 and standard deviation 1. Conversely, a standard normal value can be converted back to the original scale:

A z-score indicates how many standard deviations an observation lies above or below the mean. Probabilities are obtained from a standard normal table, calculator or statistical software.

The empirical rule is an approximation for a normal distribution: approximately 68% of observations lie within 1 standard deviation of the mean, 95% within 2, and 99.7% within 3. These percentages are descriptive guides, not universal laws.

Worked example. Delivery time is normally distributed with mean 40 minutes and standard deviation 6 minutes. Find the probability that a delivery takes less than 49 minutes.

The standard normal CDF gives P(Z ≤ 1.50) ≈ 0.9332. Therefore, approximately 93.32% of deliveries take less than 49 minutes under the model.

For a continuous variable, P(X = 49) = 0, but P(X ≤ 49) is meaningful. If times are recorded to the nearest minute, a continuity correction may be used when approximating a discrete count or rounded measurement.

Normal Approximation to a Binomial Distribution

For a binomial variable X with mean np and variance np(1 − p), a normal approximation may be reasonable when both np and n(1 − p) are sufficiently large. A common practical rule is that both are at least 10, although context matters. The approximation uses:

For a probability such as P(X ≤ k), a continuity correction uses P(Y ≤ k + 0.5), where Y is the approximating normal variable. The correction recognises that a discrete probability at k represents an interval around k.

4. The Law of Large Numbers and the Central Limit Theorem

Law of Large Numbers

The law of large numbers states that, under suitable conditions, the sample average becomes closer to the population mean as the sample size increases. If X₁, X₂, …, Xₙ are independent observations from a population with finite mean μ, then the sample mean is:

As n increases, X̄ converges in probability to μ. This does not mean that every large sample mean equals μ, nor does it eliminate unusual samples. It means that large samples tend to produce more stable averages.

Central Limit Theorem

The central limit theorem (CLT) states that, for independent observations from a population with mean μ and finite variance σ², the standardised sample mean approaches a standard normal distribution as n becomes large:

Equivalently:

The CLT is powerful because the original population need not be normal. However, the required sample size depends on skewness, heavy tails, dependence and outliers. A strongly skewed population may require a larger sample than a nearly symmetric population. Independence is especially important; a large but dependent dataset does not automatically satisfy the usual CLT conditions.

The law of large numbers concerns convergence of the sample average to μ. The CLT concerns the shape and scale of the sample-mean distribution and supports approximate probability calculations and confidence intervals. They are related but not interchangeable.

5. Sampling Distributions and Standard Errors

A sampling distribution is the probability distribution of a statistic across all possible samples of a specified size from a population. A statistic is a function of sample data, such as X̄, a sample proportion p̂ or a sample variance s².

Sampling Distribution of the Sample Mean

If a random sample of size n is drawn from a population with mean μ and standard deviation σ, then:

and, under independence,

The standard deviation of the sampling distribution is called the standard error of the mean:

If σ is unknown, it is commonly estimated using the sample standard deviation s:

Increasing n reduces the standard error by the square-root rule. To halve the standard error, the sample size must be multiplied by four. This is a reduction in sampling variability, not a guarantee of removal of bias.

If sampling is without replacement from a finite population of size N and the sample is not negligible relative to N, a finite population correction may be used:

The correction is applicable when the sampling fraction is sufficiently large and the sampling design is genuinely without replacement.

Sampling Distribution of a Sample Proportion

Suppose each observation is classified as success or failure, and p̂ is the sample proportion of successes:

For a binomial sample:

Thus:

In practice p is often unknown, so the estimated standard error is:

A normal approximation for p̂ is generally reasonable when the expected numbers of successes and failures are sufficiently large, commonly when np and n(1 − p) are both at least 10.

Sampling Distribution of a Difference in Means

For two independent samples with means X̄₁ and X̄₂, population means μ₁ and μ₂, standard deviations σ₁ and σ₂, and sample sizes n₁ and n₂:

The formula assumes independent samples. If observations are paired, a paired-difference analysis is usually more appropriate.

6. A Practical Workflow for Distribution-Based Analysis

A reliable analysis follows a sequence rather than selecting a distribution solely because it is familiar.

1. Define the random variable. Specify exactly what is being counted or measured and the unit of observation.
2. Identify the data type. Decide whether the variable is discrete, continuous, binary, a count, a proportion or a measurement.
3. Check the sampling or process design. Consider independence, replacement, fixed intervals, constant rates and possible clustering.
4. Estimate or obtain parameters. Use historical data, a justified benchmark or a pilot sample.
5. Choose the model. Use binomial for fixed independent binary trials, Poisson for event counts at a stable rate, normal for suitable continuous measurements or approximations, and hypergeometric for finite sampling without replacement.
6. Calculate using appropriate software or tables. Record parameter values and assumptions so the result is reproducible.
7. Validate the model. Inspect histograms, rates over time, outliers, dependence and goodness-of-fit evidence.
8. Interpret in context. State probability, time frame, population and practical meaning; do not present a model estimate as certainty.

7. Applied Case: Monitoring Customer-Service Demand

Background

A fictional online retailer operates a customer-service centre. Historical records suggest an average of 4 chat requests per five-minute interval during weekday afternoons. Management wants to schedule staff without excessive idle capacity or unacceptable waiting times.

Problem

The manager asks for the probability of receiving more than 6 requests in a five-minute interval and wants to estimate the reliability of a sample-based average observed over 36 intervals.

Analysis and Approach

Assume, initially, that requests follow a Poisson distribution with λ = 4 per interval. The required probability is:

Using the Poisson CDF, P(X ≤ 6) ≈ 0.8893, so:

There is approximately an 11.1% chance of more than six requests in a five-minute interval under the model.

For a Poisson variable, the standard deviation of an individual interval is √λ = √4 = 2. If the 36 intervals are independent and have the same rate, the standard error of their mean is:

The sampling distribution of the mean is approximately normal by the CLT. A mean of 4.8 requests would have an approximate z-score:

This is relatively high under the assumed rate and may prompt investigation.

Limitations and Outcome

The Poisson assumption may fail if afternoon demand varies systematically by day, if requests arrive in bursts, or if one unresolved issue generates multiple chats. The centre should plot counts by time and day, test for overdispersion, and consider a time-varying-rate model if variance substantially exceeds the mean. Staffing decisions should also incorporate service time and queue dynamics; the probability of request counts alone does not determine waiting time.

The appropriate managerial outcome is therefore conditional: use the Poisson calculation as a transparent baseline, monitor actual performance, and revise the model when evidence of changing rates or dependence appears.

Learning Points

• A distribution is a model of a process, not a substitute for understanding the process.
• The same data can support both an individual-count analysis and a sampling-distribution analysis.
• Standard errors quantify sampling variation but do not correct systematic bias or model misspecification.
• Operational decisions should combine probability results with capacity, cost and service-level information.

8. Challenges, Ethics and Technology

Common analytical errors

A frequent error is treating a probability as a guarantee. Another is assuming normality merely because a variable is numerical. Analysts may also confuse the standard deviation of individual observations with the standard error of a sample mean. Dependence, seasonality, non-response, measurement error and selection bias can make formula-based conclusions misleading.

Large samples do not automatically produce valid results. A very large biased sample can estimate the wrong target with high precision. Similarly, a large administrative dataset may contain repeated observations from the same customers, violating independence.

Ethical and governance considerations

Probability models can influence credit decisions, staffing, insurance, medical triage and customer treatment. Analysts should document data sources, assumptions, exclusions and uncertainty. Models should be tested for disparate error rates across relevant groups, and personal data should be collected and used lawfully. Automated decisions should be explainable at a level appropriate to their consequences, with human review for high-impact cases. Reporting a precise probability without disclosing model uncertainty can create false confidence.

Technology and contemporary practice

Statistical software can calculate exact and cumulative probabilities, simulate sampling distributions, estimate parameters and produce diagnostic plots. Simulation is particularly useful when assumptions are complicated: repeatedly generating samples from a specified model can illustrate the law of large numbers and CLT. However, software does not decide whether independence, constant rates or representative sampling are credible. Reproducible code, versioned data, documented random seeds where appropriate, and independent validation improve reliability. Machine-learning systems also rely on probabilistic ideas, but predictive performance does not by itself establish that a particular probability distribution or causal interpretation is correct.

Best-Practice Checklist

• Define the random variable and its unit clearly.
• Match the distribution to the data-generating mechanism.
• Check independence, stability of rates and sampling design.
• Distinguish population parameters from sample statistics.
• Report standard errors with the relevant sample size and assumptions.
• Use continuity corrections when applying a normal approximation to discrete counts.
• Validate assumptions with plots and domain knowledge.
• Treat probabilities as conditional on a model and population.
• Communicate uncertainty and practical consequences.
• Preserve an auditable record of data, calculations and decisions.

Chapter Summary

Probability distributions describe uncertainty in random variables. Discrete variables are represented by probability mass functions, while continuous variables use density functions and interval probabilities. The binomial distribution models successes in a fixed number of independent binary trials; the Poisson distribution models counts at a stable average rate; and the hypergeometric distribution models successes in sampling without replacement. The normal distribution is a central model for continuous measurements and an approximation for suitable binomial counts.

The law of large numbers explains why sample averages become more stable and approach the population mean as sample size increases. The central limit theorem explains why the distribution of a sample mean is often approximately normal, with standard deviation σ/√n. This standard deviation is the standard error and measures sampling variability, not bias. Sampling distributions support probability calculations, confidence intervals and evidence-based decisions, but their validity depends on design, independence and appropriate model assumptions. Professional analysts combine formulas and software with diagnostics, ethical governance and careful communication.

Key Terms

Random variable: A numerical value assigned to an outcome of a random process.

Discrete random variable: A variable with countable possible outcomes.

Continuous random variable: A variable that can take values throughout an interval.

Probability mass function: A function assigning probabilities to values of a discrete variable.

Probability density function: A function describing probability concentration for a continuous variable.

Cumulative distribution function: The probability that a variable is less than or equal to a specified value.

Expected value: The probability-weighted long-run mean of a random variable.

Variance: The expected squared deviation from the mean.

Standard deviation: The square root of variance.

Bernoulli trial: A trial with two possible outcomes.

Binomial distribution: The distribution of successes in fixed independent Bernoulli trials.

Poisson distribution: A distribution for event counts in a fixed interval at a stable average rate.

Normal distribution: A symmetric continuous distribution determined by its mean and standard deviation.

z-score: The number of standard deviations an observation is from a mean.

Sampling distribution: The distribution of a statistic over repeated samples.

Standard error: The standard deviation of a sampling distribution.

Law of large numbers: The principle that sample averages approach the population mean as sample size grows.

Central limit theorem: The result that standardised sample means tend toward a normal distribution under suitable conditions.

Continuity correction: An adjustment used when approximating a discrete distribution with a continuous one.

Finite population correction: An adjustment reducing sampling variance when sampling substantially without replacement from a finite population.

Review Questions

5 Short Answer Questions

1. What is the difference between a discrete and a continuous random variable?
2. State two assumptions required for a binomial model.
3. What does the Poisson parameter λ represent?
4. Define the standard error of the sample mean.
5. Distinguish the law of large numbers from the central limit theorem.

5 Long Answer Questions

1. Explain the binomial distribution, derive its mean and variance conceptually, and discuss a business situation in which its assumptions may fail.
2. Compare the binomial, Poisson and hypergeometric distributions, including their data requirements and practical applications.
3. Explain standardisation using z-scores and show how it supports probability calculations for a normal variable.
4. Describe the central limit theorem and explain why it is important for statistical inference when the population is not normal.
5. Discuss how dependence, bias and changing rates can undermine distribution-based analytics, even with a large dataset.

3 Analytical / Application Questions

1. A warehouse finds that 2% of packages are damaged. For a random sample of 20 independent packages, calculate the probability of exactly one damaged package and state whether a binomial model is appropriate.
2. A call centre receives an average of 5 calls per minute. Use a Poisson model to calculate the probability of zero calls in a one-minute interval and explain one operational implication.
3. Customer spending has population standard deviation $60. Two hundred independent customers are sampled. Calculate the standard error of the sample mean and explain how the result would change if the sample size increased to 800.

Multiple Choice Questions with Answers

Question 1

Which condition is required for a binomial distribution?

A. The number of trials is fixed and the success probability is constant.

B. The event rate must equal the variance.

C. Sampling must always be without replacement.

D. The variable must be continuous.

Correct Answer: A. The number of trials is fixed and the success probability is constant.

Question 2

A Poisson random variable has mean λ = 9. What is its standard deviation?

A. 3

B. 9

C. 81

D. √81 + 9

Correct Answer: A. 3

Question 3

If X is normally distributed with mean 50 and standard deviation 10, what is the z-score for X = 70?

A. −2

B. −0.5

C. 2

D. 7

Correct Answer: C. 2

Question 4

What does the law of large numbers state?

A. Every large sample is normally distributed.

B. A sample average tends to approach the population mean as sample size increases.

C. The population variance becomes zero in a large sample.

D. Every observation approaches the population mean.

Correct Answer: B. A sample average tends to approach the population mean as sample size increases.

Question 5

If a population standard deviation is 12 and n = 36, what is the standard error of the sample mean?

A. 0.33

B. 2

C. 6

D. 72

Correct Answer: B. 2

Question 6

Which distribution is most appropriate for the number of defective items selected without replacement from a finite batch?

A. Normal

B. Poisson

C. Hypergeometric

D. Exponential

Correct Answer: C. Hypergeometric

Question 7

For a binomial random variable with n = 100 and p = 0.20, what is its expected value?

A. 0.002

B. 5

C. 20

D. 80

Correct Answer: C. 20

Question 8

Why is a continuity correction used in a normal approximation to a discrete distribution?

A. To convert variance into standard deviation

B. To account for the gap between discrete values when using a continuous curve

C. To make the success probability constant

D. To eliminate sampling bias

Correct Answer: B. To account for the gap between discrete values when using a continuous curve

Question 9

If the sample size increases from 100 to 400 while σ remains constant, how does the standard error of the sample mean change?

A. It doubles

B. It remains unchanged

C. It becomes half as large

D. It becomes four times as large

Correct Answer: C. It becomes half as large

Question 10

For a Poisson model with λ = 4, what is the probability of zero events in the interval?

A. e⁻⁴

B. 4e⁻⁴

C. 1 − e⁻⁴

D. e⁴

Correct Answer: A. e⁻⁴