Forecast Uncertainty and Risk

Introduction

A forecast is not a promise about the future. It is an estimate conditioned on information available at a particular time. In supply chains, treating a point forecast as certain can produce stockouts, excessive inventory, unstable production schedules, and misleading performance evaluations. The central task of advanced forecasting is therefore not only to estimate expected demand, but also to describe how demand may vary and how that variation affects decisions.

Forecast uncertainty has two broad sources. Aleatory uncertainty arises from inherent randomness in demand, such as daily purchasing variation. Epistemic uncertainty arises from incomplete knowledge, such as an unobserved promotion, a structural market change, or a poor model specification. The distinction matters because additional data may reduce epistemic uncertainty, whereas aleatory uncertainty cannot be eliminated completely.

This chapter develops prediction intervals, probabilistic forecasts, quantiles, scenarios, calibration methods, tail-risk measures, and decision approaches for asymmetric business costs. The emphasis is on communicating useful ranges without false precision.

Learning Objectives

After studying this chapter, you should be able to:

1. Distinguish point forecasts, confidence intervals, prediction intervals, and probabilistic forecasts.
2. Construct and interpret prediction intervals under appropriate assumptions.
3. Use quantiles for inventory, capacity, and service decisions.
4. Design scenarios and assess tail risk.
5. Evaluate calibration, sharpness, coverage, and forecast quality.
6. Incorporate asymmetric underage and overage costs into decisions.
7. Communicate uncertainty responsibly to operational and executive audiences.

1. The Architecture of Forecast Uncertainty

Let Y_t+h denote demand at future time t+h, conditional on information available at time t, represented by I_t. A point forecast is commonly the conditional mean:

Here, E(·) is the conditional expectation. The units of μ are demand units, such as cases per week. The mean is useful for aggregate planning, but it does not reveal dispersion or tail outcomes.

A complete probabilistic forecast specifies a conditional distribution:

where F is the cumulative distribution function. It answers questions such as, “What is the probability that demand will be no greater than 1,200 units?” A probability mass function is used for discrete demand; a probability density function is used for continuous demand.

Important distinctions

A prediction interval is wider than a confidence interval when future-observation noise is material. Confusing the two can lead managers to underestimate operational risk.

2. Prediction Intervals

Assume, initially, that forecast errors are approximately normally distributed with conditional mean zero and standard deviation σ_t+h|t. A two-sided prediction interval with nominal coverage 1 − α is:

Here, z_1−α/2 is the corresponding standard-normal quantile. For 95% nominal coverage, z_0.975 ≈ 1.96. The interval is expressed in demand units. It is appropriate only when the conditional error distribution is adequately represented by the assumed model and when the forecast variance is correctly estimated.

If uncertainty comes from estimating model parameters as well as from future random demand, a more complete variance may be written as:

In many elementary applications the covariance term is ignored and parameter uncertainty is neglected. For short-horizon operational forecasting with large historical samples, that simplification may be reasonable; for long-horizon or structurally changing environments, it may not be.

Forecast-error variance over multiple periods

For a simple model with independent one-period errors of variance σ², the variance of the sum of h future errors is:

and the standard deviation is σ√h. This square-root relationship requires uncorrelated errors with constant variance. If errors are autocorrelated, the correct expression is:

Positive autocorrelation widens aggregate uncertainty; negative autocorrelation can reduce it. Forecast systems should not mechanically apply σ√h when residual diagnostics indicate dependence.

Worked example: a prediction interval

A distributor forecasts weekly demand of 10,000 units. Historical one-week forecast errors have an estimated standard deviation of 800 units. Under an approximately normal and unbiased error assumption, a 95% prediction interval is:

Thus, the interval is approximately 8,432 to 11,568 units. This does not mean that demand must fall in the interval, nor that there is a 95% probability that the fixed future demand “contains” the forecast. It means that, under the model and repeated comparable conditions, approximately 95% of constructed intervals would cover the realized observation.

If demand is nonnegative and skewed, a normal interval may produce implausible negative values or understate high-demand risk. Logarithmic, gamma, negative-binomial, bootstrap, quantile, or simulation-based approaches may be more suitable.

3. Quantiles and Decision-Relevant Forecasts

The p-quantile is defined as:

It is the smallest value at which the cumulative probability reaches p. The median is q_0.5. The 90th percentile, q_0.9, is a value exceeded in approximately 10% of comparable outcomes.

Quantiles are especially important because supply-chain decisions often have asymmetric costs. Suppose a firm chooses order quantity Q before observing demand D. Let C_u be the unit underage cost of ordering too little and C_o the unit overage cost of ordering too much. Under the classical newsvendor assumptions—single decision, known demand distribution, linear costs, no salvage complications beyond those included in C_o, and rational risk-neutral optimization—the cost-minimizing quantity is:

The critical fractile is Cu/(Cu+Co). If shortage cost is high relative to excess cost, the target quantile rises.

For example, if C_u = 12 monetary units and C_o = 4, then:

The decision should target the 75th percentile, not necessarily the mean. This result is not a universal “service-level formula”; it depends on the stated cost structure and decision setting.

In periodic-review inventory systems, demand during the protection period includes lead time and review time. If μ_L is expected demand during that period and q_p is the relevant quantile of protection-period demand, a base-stock target can be expressed as:

Safety stock is then often represented as S_p − μ_L, provided the units and timing are consistent. This quantity is not automatically equal to a fixed multiple of standard deviation when demand is skewed, intermittent, or correlated.

4. Probabilistic Forecasts and Scenario Design

A probabilistic forecast can be represented in several forms:

• A full distribution, such as a simulated sample of future demand paths.
• A set of quantiles, such as the 10th, 50th, and 90th percentiles.
• Prediction intervals at several coverage levels.
• A discrete probability tree.
• A scenario set with probabilities and explicit assumptions.

A scenario is not merely a high, medium, or low number. It should identify causal assumptions, timing, affected variables, and internal consistency. For example, a severe port disruption may increase lead time, reduce available supply, increase expedite cost, and alter customer substitution. Treating each variable independently can create impossible combinations.

A useful scenario structure includes:

Probabilities should not be invented merely to make scenarios look precise. When evidence is weak, ordinal labels such as plausible, severe, or extreme may be more honest, provided decision thresholds are still specified.

5. Calibration, Sharpness, and Forecast Evaluation

A probabilistic forecast is calibrated when predicted probabilities agree with long-run frequencies. If many events are assigned probability 0.2, approximately 20% of those events should occur, assuming comparable conditions and a sufficiently large sample.

For an interval forecast with nominal coverage 1−α, empirical coverage is:

where n is the number of forecast cases, y_t is realized demand, L_t and U_t are lower and upper limits, and the indicator equals 1 when the observation lies inside the interval. A 90% interval should have coverage near 90% over a relevant evaluation sample, although sampling variation is expected.

Calibration alone is insufficient. An interval that spans nearly every possible demand value may have excellent coverage but little operational value. Sharpness describes concentration: narrower predictive distributions are sharper, subject to being calibrated. A useful forecast is calibrated and as sharp as the data and model permit.

Proper scoring rules

Probabilistic forecasts should be evaluated with scoring rules that reward honest probability statements. For a binary event with predicted probability p and outcome o ∈ {0,1}, the Brier score is:

Lower values are better. For continuous demand, the continuous ranked probability score (CRPS) compares the entire predictive distribution with the realized observation. In practice, organizations may also use weighted interval scores, pinball loss for quantiles, and service-specific cost measures.

For a predicted quantile q_p and realized value y, pinball loss is:

Equivalent piecewise interpretation: underprediction is weighted by p, while overprediction is weighted by 1−p. Lower average loss indicates better quantile forecasts. Quantile forecasts should be evaluated at the quantiles actually used in decisions.

6. Tail Risk and Extreme Outcomes

Tail risk concerns relatively unlikely outcomes with disproportionately large consequences. In supply chains, examples include demand surges, supplier failure, prolonged disruption, regulatory restrictions, and correlated regional shortages.

A percentile alone does not fully describe tail risk. Two distributions can share the same 95th percentile while differing greatly beyond it. Useful measures include:

• Exceedance probability: P(D > d), the probability demand exceeds threshold d.
• Value at Risk (VaR): VaR_p = q_p, a percentile of loss or demand exposure.
• Expected Shortfall (ES): the expected loss conditional on exceeding a specified quantile.

For a loss variable L and continuous distribution, expected shortfall at level p is commonly written:

This definition assumes the upper tail represents adverse outcomes and requires adequate treatment of discrete distributions and ties in implementation. Expected shortfall is informative because it measures the average severity of outcomes beyond the threshold, whereas VaR does not.

Tail estimates are statistically fragile. Extreme observations are scarce, historical relationships may fail during crises, and aggregation can conceal dependence. Stress testing, expert judgment, and structural scenarios should supplement statistical extrapolation.

7. Asymmetric Costs and Risk Preferences

The optimal forecast depends on the decision loss function. A mean forecast is optimal under squared-error loss, but not under all business objectives. For absolute-error loss, the median is optimal. For asymmetric linear loss, a quantile is optimal.

Supply-chain costs may include:

• Lost margin and customer dissatisfaction from stockouts.
• Obsolescence, markdowns, storage, and working capital from excess inventory.
• Overtime, expediting, and capacity disruption.
• Contract penalties and service-level violations.
• Strategic costs from losing a customer or damaging trust.

These costs may be nonlinear. A shortage of 100 units may be manageable, while a shortage that interrupts a production line may have a much larger marginal cost. In such settings, the business should model the decision loss directly rather than simply increasing a service factor.

Risk preference also matters. A risk-neutral model minimizes expected cost. A risk-averse organization may penalize variability or tail loss. One possible objective is:

where C is total cost, Risk(C) may be variance, expected shortfall, or another approved measure, and λ ≥ 0 expresses risk aversion. This is a managerial modeling choice, not a statistical law. It must be governed and periodically reviewed.

8. Illustrative Case: Medical-Device Distribution

Background

The following is an illustrative case. MedSure distributes a temperature-sensitive diagnostic cartridge with a two-week supplier lead time. Demand is affected by hospital testing programs, respiratory disease incidence, and local tenders. Stockouts can delay patient testing; excess inventory expires after six weeks.

Problem

The planning team uses a weekly point forecast of 4,000 units and orders close to that value. During demand surges, service falls sharply. During quiet periods, expired inventory increases. Executives request a single “most accurate” forecast, but the supply-chain manager argues that the decision requires a distribution and explicit cost assumptions.

Analysis

The team segments demand by hospital cluster and estimates a probabilistic model using recent demand, tender calendars, seasonality, and epidemiological indicators. Backtesting reveals that the original normal intervals have only 72% coverage for nominal 90% intervals, with particularly poor performance during surges. Residuals are right-skewed and demand across hospital clusters is positively correlated during regional outbreaks.

The team therefore combines three elements: quantile regression for routine demand, an outbreak scenario for correlated increases, and a bootstrap of historical forecast errors. It reports the following two-week protection-period distribution:

The underage cost is estimated at 18 monetary units per cartridge, including emergency procurement and expected service damage. Overage cost is 6 units, including expiry and handling. The implied critical fractile is:

The procurement target should therefore be approximately the 75th percentile of the relevant protection-period distribution. Because the table does not list that percentile, the team obtains it directly from the forecast simulation rather than interpolating casually from only two displayed values. It also establishes a contingency trigger at the 97.5th percentile and a supplier-expedite agreement.

Approach and outcome

The dashboard presents the median, the 75th-percentile decision quantity, a 90% interval, the outbreak scenario, and the cost consequences of missing each threshold. After implementation, the organization reports fewer emergency shipments and lower expiry. More importantly, managers stop treating the median as a commitment and use the 75th percentile when shortage consequences justify it.

The result is not proof that the new method will always outperform the old one. The team continues rolling-origin backtesting, monitors calibration by hospital cluster, and revises costs when expiry or service penalties change.

Learning points

1. A point forecast cannot represent asymmetric operational consequences.
2. Calibration must be checked during unusual as well as ordinary periods.
3. Correlation across locations can make aggregate tail risk much larger than independent-location calculations suggest.
4. A quantile is a decision input, not a universal inventory answer.
5. Communication improves when ranges are linked to actions and cost consequences.

9. Governance, Technology, and Responsible Communication

Forecast uncertainty should be governed as carefully as point forecasts. Every published interval should have a horizon, unit, confidence or coverage interpretation, data vintage, model version, and known limitations. Forecast revisions should be traceable so that users can distinguish new information from silent model changes.

Artificial intelligence and machine-learning systems can produce probabilistic forecasts through quantile regression, distributional neural networks, ensembles, Bayesian models, and simulation. Their complexity does not guarantee calibration. They require temporal validation, leakage controls, monitoring for concept drift, and subgroup analysis. A model that is well calibrated overall may be poorly calibrated for small customers, remote regions, or high-value products.

Human judgment is valuable when structural breaks, policy changes, and one-off events are not represented in historical data. However, judgmental overrides should record the rationale, evidence, owner, and expiration date. Unrecorded overrides make learning and accountability difficult.

Communication should avoid false precision. A statement such as “demand will be 10,037 units” implies a level of certainty usually unsupported by the data. A stronger communication is: “The median is 10,000 units; the 80% prediction interval is 8,700–11,600; the recommended order quantity is 11,100 because shortage cost is estimated to be twice excess cost.” The range, decision rule, and assumptions are all visible.

10. Best-Practice Framework

A mature uncertainty-management process should:

1. Define the decision, horizon, unit, and relevant exposure.
2. Separate forecast uncertainty from parameter and scenario uncertainty.
3. Produce distributions or quantiles rather than only point estimates.
4. Test assumptions for skewness, autocorrelation, heteroscedasticity, and dependence.
5. Backtest using rolling time origins and realistic information availability.
6. Evaluate calibration, sharpness, proper scores, and business costs.
7. Model asymmetric costs and identify the decision quantile.
8. Stress-test correlated disruptions and tail outcomes.
9. Link ranges to triggers, mitigations, and ownership.
10. Monitor performance and revise models as the environment changes.

Chapter Summary

Forecast uncertainty is an essential component of supply-chain decision-making. Point forecasts describe a central estimate, whereas probabilistic forecasts describe possible outcomes and their probabilities. Prediction intervals must be distinguished from confidence intervals and interpreted under explicit assumptions. Quantiles are often more useful than means when underage and overage costs are asymmetric; the classical newsvendor model selects the quantile determined by the critical fractile.

Reliable probabilistic forecasting requires calibration, sharpness, suitable scoring rules, and realistic treatment of dependence and structural change. Tail risk requires attention to both the probability and severity of extreme outcomes, with expected shortfall complementing percentile-based measures. Scenarios are most useful when they are causally coherent and connected to operational triggers. Finally, responsible communication reports ranges, assumptions, and decision implications rather than false precision.

Key Terms

Aleatory uncertainty: Random variation inherent in the process being forecast.

Calibration: Agreement between predicted probabilities and observed frequencies.

Confidence interval: An interval concerning uncertainty in an estimated parameter or mean.

Critical fractile: The cost-based probability target C_u/(C_u+C_o) in the classical single-period inventory model.

Epistemic uncertainty: Uncertainty caused by incomplete knowledge or model limitations.

Expected shortfall: Average loss conditional on loss exceeding a specified percentile.

Forecast distribution: Conditional probability distribution of future outcomes.

Overage cost: Cost resulting from ordering or producing more than is needed.

Pinball loss: An asymmetric loss function used to evaluate quantile forecasts.

Point forecast: A single numerical estimate of a future value.

Prediction interval: A range intended to contain a future observation with stated nominal coverage.

Probabilistic forecast: A forecast assigning probabilities to possible future outcomes.

Quantile: A value below which a specified proportion of outcomes falls.

Risk aversion: Preference for avoiding variability or adverse tail outcomes beyond expected cost.

Scenario: A coherent set of assumptions describing a possible future state.

Sharpness: Concentration or narrowness of a predictive distribution, subject to calibration.

Tail risk: Risk associated with relatively rare but consequential outcomes.

Underage cost: Cost resulting from ordering or producing less than is needed.

Value at Risk: A specified percentile of a loss distribution.

Review Questions

Short Answer Questions

1. What is the difference between a point forecast and a probabilistic forecast?
2. Why is a prediction interval generally wider than a confidence interval for a mean?
3. Define calibration and explain how empirical coverage is calculated.
4. What does the 90th percentile of demand represent?
5. Why can a normal prediction interval be unsuitable for highly skewed demand?

Long Answer Questions

1. Explain the assumptions underlying the standard normal prediction-interval formula and discuss situations in which those assumptions fail.
2. Derive and interpret the critical-fractile rule for a single-period inventory decision with underage and overage costs.
3. Compare calibration, sharpness, and proper scoring rules as criteria for evaluating probabilistic forecasts.
4. Explain how dependence across products, locations, or time periods affects aggregate tail risk.
5. Discuss how forecast uncertainty should be governed when machine-learning systems and human overrides are both used.

Analytical / Application Questions

1. A firm estimates an underage cost of 20 monetary units and an overage cost of 5 units. Determine the target quantile under the classical newsvendor assumptions and explain what additional information is needed to set the order quantity.
2. A nominal 90% interval covers 84 of 100 realized observations. Assess the evidence regarding calibration and identify at least three possible causes of undercoverage.
3. Design a scenario-based forecasting approach for a retailer facing a possible port closure. Specify causal assumptions, affected variables, decision triggers, and methods for evaluating whether the resulting risk communication is useful.

Multiple Choice Questions with Answers

Question 1

Which statement best distinguishes a prediction interval from a confidence interval?

A. A prediction interval concerns a future observation, while a confidence interval commonly concerns an estimated parameter or mean.

B. A prediction interval is always narrower than a confidence interval.

C. A confidence interval assigns probabilities to every individual future observation.

D. They are identical terms in supply-chain forecasting.

Correct Answer: A. A prediction interval concerns a future observation, while a confidence interval commonly concerns an estimated parameter or mean.

Question 2

Under the classical newsvendor assumptions, if the underage cost is 12 and the overage cost is 4, which quantile should be targeted?

A. 25th percentile

B. 50th percentile

C. 75th percentile

D. 90th percentile

Correct Answer: C. 75th percentile

Question 3

A forecast interval has nominal 90% coverage and contains 84 of 100 realized observations. What is the empirical coverage?

A. 8.4%

B. 16%

C. 84%

D. 90%

Correct Answer: C. 84%

Question 4

What does calibration mean for probabilistic forecasts?

A. Forecast intervals are as narrow as possible regardless of misses.

B. Predicted probabilities agree with long-run observed frequencies.

C. The point forecast always equals the realized demand.

D. All forecast errors have the same monetary cost.

Correct Answer: B. Predicted probabilities agree with long-run observed frequencies.

Question 5

Which measure focuses on the average severity of losses beyond a specified percentile?

A. Median

B. Mean absolute error

C. Expected shortfall

D. Point forecast bias

Correct Answer: C. Expected shortfall

Question 6

For independent one-period errors with variance σ², what is the variance of the sum of h future errors?

A. σ²/h

B. hσ²

C. σ√h

D. h/σ²

Correct Answer: B. hσ²

Question 7

Why can a calibrated interval still be operationally unhelpful?

A. Calibration prevents any use of quantiles.

B. A very wide interval may have good coverage but poor sharpness.

C. Calibration means the interval contains no demand observations.

D. Sharpness is relevant only to point forecasts.

Correct Answer: B. A very wide interval may have good coverage but poor sharpness.

Question 8

Which feature is essential to a coherent disruption scenario?

A. A precise probability invented without evidence

B. Independent random changes to every variable

C. Explicit causal assumptions linking the disruption to operational outcomes

D. A single point estimate with no timing information

Correct Answer: C. Explicit causal assumptions linking the disruption to operational outcomes

Question 9

Under absolute-error loss, which central forecast is generally optimal?

A. The mean

B. The median

C. The maximum

D. The 99th percentile

Correct Answer: B. The median

Question 10

What is a major governance requirement for judgmental forecast overrides?

A. They should remain undocumented to preserve flexibility.

B. They should replace all statistical forecasts.

C. Their rationale, evidence, owner, and expiration should be recorded.

D. They should be applied only after observing the actual outcome.

Correct Answer: C. Their rationale, evidence, owner, and expiration should be recorded.