Time Series Forecasting

Chapter 11
Probabilistic, Hierarchical, and Scenario Forecasting

Post Graduate Program in Business Analytics
Time Series Forecasting   •   Post Graduate

Probabilistic, Hierarchical, and Scenario Forecasting

Introduction

Point forecasts answer a narrow question: what value is expected at a future time? Business decisions, however, depend on uncertainty. A retailer must decide how much inventory to hold across possible demand outcomes; a bank must estimate losses under adverse conditions; and a supply-chain manager must assess whether regional forecasts add up to a corporate plan. These tasks require probabilistic forecasts, forecasts for related series that obey aggregate constraints, and coherent scenarios.

A probabilistic forecast represents uncertainty through quantiles, prediction intervals, or a full probability distribution. A hierarchical or grouped forecast represents several related time series and requires forecasts that are mutually consistent. Scenario forecasting describes alternative future paths arising from assumptions about economic, operational, environmental, or strategic conditions. These three perspectives are closely related: a scenario may shift a predictive distribution, while reconciliation may ensure that scenario forecasts remain consistent across business levels.

This chapter develops methods for producing, evaluating, reconciling, and communicating such forecasts. It emphasizes calibration, coverage, pinball loss, proper scoring rules, tail risk, and governance.

Learning objectives

By the end of this chapter, the reader should be able to:

  1. Distinguish point, quantile, interval, and distributional forecasts.
  2. Construct and interpret prediction intervals and quantiles.
  3. Evaluate forecasts using calibration, coverage, pinball loss, CRPS, and related proper scoring rules.
  4. Explain why independently forecast hierarchical series may be incoherent.
  5. Apply bottom-up, top-down, middle-out, and optimal-combination reconciliation.
  6. Design scenarios without confusing them with probabilities.
  7. Communicate uncertainty, aggregate constraints, and tail risks responsibly.

1. From point forecasts to predictive distributions

Let yt+h denote the value observed at forecast origin t and horizon h. Information available at the origin is represented by \mathcal{I}t. A point forecast is commonly the conditional mean:

\hat{y}t+h|t = E(Yt+h | \mathcal{I}t)

The notation E(·) denotes conditional expectation. This forecast is optimal under squared-error loss, but it does not describe uncertainty. A predictive distribution is the conditional distribution:

Ft,h(z) = P(Yt+h \le z | \mathcal{I}t)

where z is a possible future value. It can answer questions such as:

  • What is the probability that demand exceeds 10,000 units?
  • What inventory level covers demand with 95% probability?
  • How large could a loss be in an adverse but plausible outcome?

Quantiles and intervals

The \tau-quantile, for 0 < \tau < 1, is often defined as

q_\tau = F-1(\tau) = inf\{z: F(z) \ge \tau\}

A 50th percentile is a median forecast. A central 80% prediction interval can be written as [q0.10, q0.90]. Its nominal coverage is 80%, meaning that, under a correctly specified and calibrated forecasting process, approximately 80% of future observations should fall within the interval.

Intervals may be central, equal-tailed, or highest-density. Equal-tailed intervals allocate equal probability to both tails. Highest-density intervals can be shorter for skewed distributions but may be discontinuous for multimodal distributions. In business reporting, equal-tailed intervals are usually easier to explain.

A forecast interval is not the same as a confidence interval. A confidence interval describes uncertainty about an estimated parameter; a prediction interval describes uncertainty about a future observation, including future process variation.

Distributional forecasts

A distributional forecast specifies either a parametric distribution, such as a normal, Poisson, negative binomial, or log-normal distribution, or a non-parametric representation such as simulation draws. Parametric models are compact but depend on distributional assumptions. Simulation-based forecasts are flexible and can preserve nonlinear dependencies, but they require careful generation and validation.

For a continuous normal predictive distribution with mean \mu and standard deviation \sigma,

Yt+h | \mathcal{I}t \sim N(\mut,h, \sigma2t,h)

The interval between probabilities \alpha/2 and 1-\alpha/2 is

\mut,h \pm z1-\alpha/2\sigmat,h

where z1-\alpha/2 is the corresponding standard-normal quantile. This formula is appropriate only when the predictive normal assumption is defensible and uncertainty is represented by \sigmat,h. For count data, bounded data, or strongly skewed data, another distribution or a transformation may be preferable.

2. Measuring probabilistic forecast quality

Probabilistic forecasts have at least two dimensions: calibration and sharpness. Calibration asks whether predicted probabilities agree with observed frequencies. Sharpness asks whether forecasts are concentrated rather than unnecessarily diffuse. Sharpness is desirable only subject to calibration; a forecast that assigns 100% probability to a huge range is safe but uninformative.

Calibration and coverage

For an interval [lt,ut] with nominal coverage 1-\alpha, an empirical coverage estimate over n forecast cases is

\widehat{C} = \frac{1}{n}\sumi=1n I(li \le yi \le ui)

where I(·) equals 1 when the statement is true and 0 otherwise. A well-calibrated interval has \widehat{C} close to 1-\alpha over a sufficiently large and representative evaluation sample. Coverage alone is insufficient: very wide intervals can achieve high coverage while offering little decision value.

Quantile calibration requires that the proportion of observations below the predicted \tau-quantile is approximately \tau:

\frac{1}{n}\sumi=1n I(yi \le \hat{q}\tau,i) \approx \tau

Calibration should be examined by horizon, product, region, season, and regime. A forecast can have correct overall coverage but systematically under-cover during promotions or recessions.

Pinball loss

The pinball, or check, loss evaluates a quantile forecast:

L_\tau(y,q) = (\tau-I(y<q))(y-q)

Equivalently, it is \tau(y-q) when y\ge q and (\tau-1)(y-q) when y<q. Here y is the observation and q is the forecast quantile. The loss is asymmetric: underestimating a high quantile is penalized more heavily than overestimating it. Minimizing expected pinball loss identifies the true \tau-quantile under standard regularity conditions, which makes pinball loss a proper scoring rule for quantiles.

For multiple quantiles, mean weighted quantile loss can be used:

\frac{1}{K}\sumk=1K L\tauk(y,q\tauk)

where K is the number of evaluated quantile levels. Quantiles must be checked for crossing, such as a 90th percentile falling below a 50th percentile. Crossing indicates an incoherent quantile representation and can be corrected through constrained post-processing or a distributional model.

Interval scores

For a central interval [l,u] with nominal miscoverage \alpha, the interval score is

IS_\alpha(l,u;y) = (u-l) + \frac{2}{\alpha}(l-y)I(y<l) + \frac{2}{\alpha}(y-u)I(y>u)

The first term rewards narrow intervals. The second and third terms penalize observations outside the interval, with larger penalties for more serious misses. Lower scores are better. This is a proper scoring rule for central prediction intervals when the nominal level is fixed. It should be reported together with empirical coverage, because a score is easier to interpret when users know whether misses are too frequent or intervals are simply too wide.

Proper scoring rules for full distributions

A scoring rule is proper if the expected score is optimized by reporting the true predictive distribution. Properness discourages strategic distortion. One widely used score is the continuous ranked probability score (CRPS):

CRPS(F,y) = \int-\infty\infty[F(z)-I(z\ge y)]2 dz

where F is the predictive cumulative distribution and y is the realized value. Lower CRPS is better. CRPS rewards both calibration and concentration and reduces to absolute error for a degenerate point forecast. For a sample of m predictive draws x1, ..., xm, a commonly used approximation is

CRPS \approx \frac{1}{m}\sumj=1m|xj-y| - \frac{1}{2m2}\sumj=1m\sumk=1m|xj-xk|

This approximation assumes the draws represent the predictive distribution. Logarithmic score, -log f(y), is also proper when a predictive density f exists, but it can be extremely sensitive to assigning very low density to an event that occurs. Choice of score should therefore reflect the decision context and data quality.

3. Producing probabilistic forecasts

Several approaches are available.

Parametric forecasting. Estimate the parameters of a conditional distribution. Examples include Gaussian innovations for ARIMA models, count distributions for demand, and generalized linear models for intermittent demand. The benefit is interpretability; the risk is misspecification.

Quantile regression. Model a selected quantile directly using predictors such as price, promotions, calendar variables, and weather. For observation yi and feature vector xi, linear quantile regression estimates \beta_\tau by minimizing

\min\beta_\tau\sumi=1n L_\tau(yi,xiT\beta_\tau)

The method does not require constant variance or normal errors. Separate quantile models may cross, so monotonicity constraints or a common distributional model may be needed.

Distributional regression and machine learning. A model predicts parameters of a distribution, or directly generates samples. Neural networks may estimate location, scale, and shape, while tree-based methods can estimate conditional quantiles. Flexible models require time-aware validation, leakage controls, and monitoring for distribution shift.

Conformal prediction. Conformal methods construct prediction sets using calibration residuals or nonconformity scores. Under exchangeability, finite-sample marginal coverage can be guaranteed at a selected level. Time series violate ordinary exchangeability because of dependence and changing distributions; rolling, weighted, block, or adaptive conformal methods are therefore used, but their guarantees require explicit assumptions and may be marginal rather than conditional.

A practical workflow is: define the forecast origin and horizon; choose the uncertainty representation; train only on information available at the origin; generate forecasts; evaluate with rolling-origin backtesting; diagnose calibration and sharpness; and monitor performance after deployment.

4. Hierarchical and grouped forecasting

A hierarchy contains series at different aggregation levels. For example, total company sales may equal sales for regions, which equal sales for stores. A grouped structure may contain several non-nested aggregations, such as sales by region and by product category. Let yt be a vector of all series and let bt contain the bottom-level series. The aggregation relationship can be represented by a summing matrix S:

yt = S bt

The matrix contains zeros and ones for simple sums, although more general fixed coefficients are possible. A forecast vector \hat{y} is coherent if there exists a bottom-level vector \hat{b} such that

\hat{y}=S\hat{b}

Forecasting each series independently usually violates this condition. For example, a regional forecast may sum to 9,800 units while the separately produced national forecast is 10,000 units. Both may be statistically reasonable in isolation, but they cannot be used together in a single plan.

Reconciliation methods

Bottom-up reconciliation forecasts the most disaggregated series and sums them upward. It preserves detailed forecasts but can be noisy where bottom-level histories are short.

Top-down reconciliation forecasts the aggregate and allocates it using historical proportions. If pj is the share allocated to component j, then

\hat{y}j=pj\hat{y}total, \qquad \sumj pj=1

Shares may be historical averages, proportions adjusted for trends, or estimated using a model. This approach can suppress genuine changes in composition.

Middle-out reconciliation forecasts a selected intermediate level, then aggregates upward and allocates downward. It is useful when the middle level has enough history and is operationally meaningful.

Optimal combination. Let \hat{y} be a vector of base forecasts and W an estimate of their forecast-error covariance matrix. A widely used reconciliation form is

\tilde{y}=S(ST W-1S)-1ST W-1\hat{y}

This expression assumes that W is positive definite or suitably regularized and that the summing matrix has appropriate rank. It projects base forecasts onto the coherent subspace while accounting for estimated error covariance. In practice, shrinkage or diagonal approximations may be required because covariance estimates can be unstable.

Reconciliation can be applied to point forecasts and to probabilistic forecasts. For simulation-based forecasts, each draw can be reconciled, preserving a coherent joint sample. Applying reconciliation separately to quantiles is more difficult because quantile operations are nonlinear; independently reconciling quantiles can produce invalid distributions or distorted dependence. A preferred approach is often to reconcile simulated paths or estimate a coherent joint distribution.

Coherence is not accuracy

Reconciliation guarantees consistency, not superior forecasts. A coherent but biased forecast may be less accurate than an incoherent base forecast at one level. Evaluation should therefore be conducted at every important level and horizon, using business-weighted losses where appropriate. A national planning process may prioritize aggregate accuracy, while store replenishment may prioritize local service levels.

5. Scenario forecasting and tail risk

A scenario is a structured description of a possible future conditional on explicit assumptions. Examples include a baseline economy, a recession, a supply disruption, a competitor price cut, or an extreme-weather event. Scenarios are not automatically probabilities. A “downside scenario” may be deliberately conservative and need not have a 20% probability.

A useful scenario specification identifies:

  1. The triggering assumptions and their time horizon.
  2. The variables affected directly and indirectly.
  3. The causal or statistical model used to propagate effects.
  4. The resulting forecast path and uncertainty around it.
  5. Decision thresholds and pre-agreed management actions.

Scenario forecasts can be generated by changing exogenous inputs, modifying model parameters, conditioning on stress events, or simulating from a joint model. If s indexes scenarios and ps are genuine probabilities summing to one, a scenario-weighted expectation is

E(Y)=\sums=1Sps E(Y|s), \qquad \sums=1Sps=1

This equation is appropriate only when scenario probabilities are meaningful and mutually exclusive and collectively exhaustive. If scenarios are exploratory narratives, they should be presented separately rather than averaged.

Tail risk concerns unusually adverse or favorable outcomes. A quantile such as value-at-risk at level \alpha is the \alpha-quantile of a loss distribution. Expected shortfall, also called conditional value-at-risk in common applications, is the expected loss beyond a threshold, subject to a precise convention for discrete distributions. For a continuous loss variable L with quantile VaR_\alpha,

ES_\alpha = E[L \mid L \ge VaR_\alpha]

where \alpha is a high confidence level such as 0.95 or 0.99. Expected shortfall describes severity beyond the quantile, but its estimation is data-intensive and sensitive to tail assumptions.

Scenario analysis should combine statistical forecasts with expert knowledge, but experts should document assumptions rather than silently override model outputs. Stress scenarios should be severe enough to test resilience while remaining causally interpretable. “Worst case” should not be used casually: there is rarely a defensible absolute worst case in an open system.

6. Integrated illustrative case: OmniRetail demand planning

The following case is fictional and illustrative.

Background and problem

OmniRetail sells household products through four regions and two channels. Its planning team produces weekly forecasts for stores, regions, channels, and the company total. Independently generated forecasts are often incoherent. Managers also complain that a “95% forecast” sometimes misses demand during promotions, creating stockouts.

Analysis

The team establishes a rolling-origin evaluation over 52 weeks and three horizons. It collects 2,000 simulated demand paths for every bottom-level region–channel combination. Calibration checks show that nominal 80% intervals cover only 68% of observations during promotions, while non-promotion coverage is 82%. This is conditional miscalibration: overall performance hides a promotion-specific failure.

The team evaluates the 0.90 quantile with pinball loss and the full distribution with CRPS. A negative-binomial model improves performance for low-volume products, while a gradient-boosted quantile model improves promotional forecasts. However, the machine-learning model produces quantile crossing in 4% of cases, so monotonic rearrangement is applied before reporting quantiles.

Approach and solution

The organization defines the bottom-level series as region–channel–product groups. Forecast simulations are generated using promotional indicators, price, inventory availability, and weather forecasts. Each simulated path is reconciled through an optimal-combination method using a shrinkage covariance estimate. This ensures that every draw aggregates consistently to regional, channel, and company totals.

Three scenarios are prepared:

  • Baseline: approved prices, normal supply, and central economic assumptions.
  • Downside: a supplier delay, weaker consumer spending, and lower promotional response.
  • Upside: improved availability and stronger campaign effectiveness.

The downside scenario is not assigned a probability. It is used to test inventory and cash requirements. Within each scenario, probabilistic paths quantify residual uncertainty. The planning dashboard reports median demand, 80% and 95% intervals, probability of exceeding capacity, and expected shortage units.

Outcome

After two quarters, aggregate forecasts become coherent by construction. Promotional coverage improves after adding interaction terms and recalibrating the upper quantiles. Managers stop interpreting the downside scenario as a probabilistic prediction and instead use it as a contingency trigger. Evaluation shows that the reconciled forecasts slightly worsen accuracy for some small stores but improve company-level accuracy and eliminate contradictory plans.

Learning points

The case demonstrates that calibration must be conditional on relevant regimes, coherence is a structural requirement rather than a measure of accuracy, and scenario narratives should be separated from probability-based predictive distributions. It also shows why simulated paths are useful: they allow reconciliation while preserving uncertainty and support capacity-risk calculations.

7. Governance, ethics, and implementation risks

Probabilistic forecasts can create false confidence when users mistake a calibrated historical interval for a guarantee. Forecast ranges should state the horizon, population, evaluation period, and assumptions. Tail estimates based on few observations should be labeled as uncertain rather than presented with excessive precision.

Hierarchical systems can distribute error unevenly. Reconciliation may cause a small region to absorb changes needed to make a national total coherent. Allocation rules should therefore be transparent, and affected business units should be able to challenge implausible outputs.

Scenario design can encode managerial or political bias. Organizations should record who created each scenario, which assumptions changed, and whether scenarios overlap. Sensitive forecasts involving credit, staffing, or public services require fairness checks across relevant groups. Personal data should be minimized, access controlled, and retained only as long as justified.

Operational best practice includes versioned data and models, reproducible forecast origins, backtesting before deployment, monitoring of coverage and scoring rules, alert thresholds for drift, and human review for exceptional events. Artificial intelligence does not remove these requirements. Complex models still require explainability appropriate to the decision, uncertainty estimates, and controls against leakage and synthetic confidence.

8. Contemporary developments and research directions

Recent work increasingly emphasizes probabilistic deep learning, foundation models for time series, diffusion-based path generation, probabilistic transformers, and conformal methods adapted to nonstationary data. These methods may improve flexibility, but benchmark gains do not establish reliability under structural breaks or rare events. Research attention is also moving toward multivariate calibration, coherent probabilistic forecasting, decision-focused scoring, and distributional shift detection.

A central methodological issue is dependence. Marginal intervals for each product may be individually calibrated but fail to represent joint events, such as all regions experiencing a shortage simultaneously. Copulas, multivariate state-space models, factor models, and reconciled simulations can model dependence. The right approach depends on the decision: a store-level reorder may need a marginal distribution, whereas cash and capacity planning may require joint tail probabilities.

Chapter Summary

Probabilistic forecasting extends point forecasting by representing uncertainty through quantiles, intervals, or full predictive distributions. Calibration measures agreement between predicted probabilities and observed frequencies, while sharpness rewards informative concentration. Pinball loss evaluates quantiles, interval score evaluates prediction intervals, and CRPS evaluates full distributions; proper scoring rules discourage strategic forecasts.

Hierarchical and grouped forecasts must satisfy aggregation constraints. Bottom-up, top-down, middle-out, and optimal-combination methods provide different trade-offs between detail, stability, and statistical efficiency. Reconciliation creates coherence but does not guarantee accuracy. Simulation-based reconciliation is particularly useful for coherent probabilistic forecasts.

Scenarios are conditional future narratives or model-based alternatives, not necessarily probability-weighted predictions. They should document assumptions, distinguish baseline uncertainty from stress conditions, and support decisions such as capacity, inventory, and capital planning. Tail measures such as high quantiles and expected shortfall help communicate extreme risks, but they are sensitive to data and modeling assumptions. Effective implementation requires rolling-origin evaluation, conditional calibration checks, transparent governance, and careful communication.

Key Terms

Probabilistic forecast: A forecast represented by a probability distribution or set of probability statements.

Quantile: A value below which a specified proportion of a distribution lies.

Prediction interval: A range intended to contain a future observation with a stated probability.

Calibration: Agreement between forecast probabilities and observed frequencies.

Sharpness: Concentration of a predictive distribution, assessed subject to calibration.

Pinball loss: An asymmetric proper scoring rule for quantile forecasts.

Interval score: A proper score combining interval width and penalties for misses.

CRPS: A proper scoring rule for evaluating a full predictive distribution.

Proper scoring rule: A scoring rule whose expected value is optimized by truthful forecasting.

Hierarchy: A set of time series related through aggregation levels.

Grouped series: Related series with multiple, possibly non-nested, aggregation structures.

Summing matrix: A matrix that maps bottom-level series to all hierarchical series.

Coherence: Exact consistency between forecasts at different aggregation levels.

Reconciliation: Adjustment of base forecasts to satisfy aggregation constraints.

Bottom-up method: A method that forecasts bottom-level series and aggregates upward.

Scenario: A structured conditional description of a possible future.

Tail risk: Risk associated with unusually extreme outcomes.

Expected shortfall: Average loss beyond a specified high loss quantile.

Distribution shift: A change in the data-generating process after model development.

Review Questions

Short Answer Questions

  1. What is the difference between a prediction interval and a confidence interval?
  2. Define calibration and sharpness in probabilistic forecasting.
  3. Why is pinball loss asymmetric?
  4. What does it mean for hierarchical forecasts to be coherent?
  5. Why should a downside scenario not automatically be interpreted as a 20% probability event?

Long Answer Questions

  1. Explain how quantile forecasts, interval forecasts, and distributional forecasts differ, and identify suitable evaluation methods for each.
  2. Derive the empirical coverage measure for a prediction interval and discuss why coverage alone is insufficient.
  3. Compare bottom-up, top-down, middle-out, and optimal-combination reconciliation methods.
  4. Explain the role of proper scoring rules in discouraging strategically distorted probabilistic forecasts.
  5. Discuss how an organization should govern scenario design, tail-risk communication, and model monitoring.

Analytical / Application Questions

  1. A 90% interval covers 86 of 100 observations and is much wider than a competitor’s interval, which covers 78 observations. Analyze the calibration–sharpness trade-off and recommend further tests.
  2. A company forecast is 10,000 units, while four regional forecasts sum to 9,600 units. Explain two ways to reconcile the forecasts and discuss the operational consequences of each.
  3. Design a baseline, downside, and upside forecasting framework for a retailer facing a possible supply disruption. Specify which assumptions are scenario inputs, which uncertainties remain probabilistic, and how tail risk should be reported.

Multiple Choice Questions with Answers

Question 1

Which property asks whether predicted probabilities agree with observed frequencies?

A. Sharpness

B. Calibration

C. Coherence

D. Stationarity

Correct Answer: B. Calibration

Question 2

Which scoring rule is specifically designed to evaluate a quantile forecast?

A. Pinball loss

B. Mean squared error only

C. Akaike information criterion

D. Durbin–Watson statistic

Correct Answer: A. Pinball loss

Question 3

An 80% prediction interval is evaluated over 200 cases and contains 156 observations. What is its empirical coverage?

A. 68%

B. 72%

C. 78%

D. 80%

Correct Answer: C. 78%

Question 4

If regional forecasts sum to a value different from the national forecast, the set of forecasts is:

A. Sharp

B. Calibrated

C. Coherent

D. Incoherent

Correct Answer: D. Incoherent

Question 5

Which reconciliation method forecasts the most disaggregated series and aggregates them upward?

A. Top-down

B. Bottom-up

C. Middle-out

D. Scenario weighting

Correct Answer: B. Bottom-up

Question 6

What is the main purpose of the width term in the interval score?

A. To reward narrow intervals

B. To estimate autocorrelation

C. To enforce seasonal adjustment

D. To calculate a point forecast

Correct Answer: A. To reward narrow intervals

Question 7

Which statement about a scenario is correct?

A. It must always have a precisely estimated probability

B. It is necessarily the same as a confidence interval

C. It is a conditional description of a possible future

D. It can contain no assumptions

Correct Answer: C. It is a conditional description of a possible future

Question 8

For a continuous loss distribution, expected shortfall at a high level describes:

A. The median loss

B. The average loss beyond a high loss quantile

C. The narrowest prediction interval

D. The probability of zero loss

Correct Answer: B. The average loss beyond a high loss quantile

Question 9

Why can reconciling simulated forecast paths be preferable to reconciling quantiles independently?

A. It removes all uncertainty

B. It preserves coherent joint samples

C. It guarantees zero bias

D. It eliminates the need for validation

Correct Answer: B. It preserves coherent joint samples

Question 10

Which score evaluates an entire predictive distribution rather than only one quantile?

A. CRPS

B. MAPE only

C. Classification accuracy

D. The sample mean

Correct Answer: A. CRPS
Time Series Forecasting
Chapter 11 of 15