9. Time-Series Forecasting and Sequential Prediction

Introduction and Context

Many organisational decisions depend not only on the value of a variable, but also on when that value occurs. Electricity demand, sales revenue, website traffic, exchange rates, inventory consumption, hospital admissions and sensor readings are recorded in temporal order. Such data form a time series. Forecasting a time series involves using information available up to a particular time to estimate one or more future observations.

Time-series forecasting differs from ordinary supervised learning because observations are usually dependent across time. A random shuffle can place future information in the training set and past information in the test set, producing an unrealistically optimistic evaluation. Forecasting therefore requires attention to temporal ordering, changing distributions, delayed effects, seasonality and operational constraints.

This chapter develops a practical and analytical framework for forecasting. It examines trend, seasonality, stationarity and autocorrelation; introduces baseline, exponential-smoothing and ARIMA models; explains machine-learning approaches to sequential prediction; and presents validation methods that reduce temporal leakage. The emphasis is on choosing and evaluating models rather than assuming that a more complex algorithm is automatically better.

Learning Objectives

After studying this chapter, the learner should be able to:

1. Distinguish trend, seasonality, cycles, noise and structural change in time-series data.

2. Explain stationarity, autocorrelation and partial autocorrelation.

3. Construct naïve and seasonal-naïve forecasting baselines.

4. Apply moving averages, exponential smoothing, Holt and Holt–Winters methods.

5. Interpret the principal components of ARIMA and seasonal ARIMA models.

6. Transform a time series into supervised-learning data without temporal leakage.

7. Design rolling-origin validation and select suitable forecast metrics.

8. Compare statistical and machine-learning forecasting approaches in professional settings.

9. Recognise limitations, governance concerns and operational risks in forecasting systems.

The Structure of Temporal Data

A useful conceptual decomposition represents an observed series as the combination of systematic and irregular components. In an additive representation,

\[y_t = T_t + S_t + C_t + \varepsilon_t\]

where \(T_t\) is trend, \(S_t\) is seasonal variation, \(C_t\) is a longer-term cycle, and \(\varepsilon_t\) is irregular noise. A multiplicative representation is more appropriate when seasonal fluctuations grow with the level of the series:

\[y_t = T_t \times S_t \times C_t \times \varepsilon_t.\]

Trend describes persistent upward or downward movement. Seasonality is a repeating pattern with a known or reasonably stable period, such as higher retail demand every December or increased electricity consumption during working hours. A cycle is a broader fluctuation whose length is not necessarily fixed, such as an economic expansion and contraction. Noise represents variation that cannot be reliably predicted from the available information.

A crucial distinction is between seasonality and a calendar effect. A weekly pattern may arise because of weekdays and weekends, while a holiday effect may occur on dates that move relative to the calendar. Organisations should encode such effects explicitly when they are relevant.

Lagged Dependence and Autocorrelation

Autocorrelation measures the relationship between observations separated by a lag. The lag-one autocorrelation compares \(y_t\) with \(y_{t-1}\); lag twelve may be especially informative for monthly data with annual seasonality. Positive autocorrelation means that high values tend to follow high values, and low values tend to follow low values. Negative autocorrelation may indicate alternating behaviour.

The autocorrelation function (ACF) displays correlations at several lags. The partial autocorrelation function (PACF) measures the relationship at a particular lag after controlling for shorter lags. ACF and PACF plots are useful diagnostic tools, although they should support—not replace—substantive understanding and out-of-sample validation.

Stationarity

A weakly stationary series has a stable mean, stable variance and autocovariance that depends on the lag rather than on the calendar time itself. Many classical models work more effectively when the series, or a transformed version of it, is stationary. A trend or changing variance can violate this condition.

Differencing is a common transformation:

\[\nabla y_t = y_t-y_{t-1}.\]

For seasonal differencing with period \(m\),

\[\nabla_m y_t = y_t-y_{t-m}.\]

Logarithmic or Box–Cox transformations can stabilise variance. However, differencing should not be applied mechanically. Excessive differencing can remove meaningful structure and increase forecast uncertainty. Stationarity tests, such as the augmented Dickey–Fuller test and KPSS test, provide evidence but are sensitive to sample size, structural breaks and model assumptions. Plots, domain knowledge and residual diagnostics remain essential.

Forecasting as a Decision Process

A forecasting project begins with the decision to be supported. A model for workforce scheduling may require hourly forecasts for the next two days, whereas a capital-budgeting process may require monthly forecasts for several years. The forecast horizon, frequency, aggregation level, cost of errors and availability of predictors determine the appropriate method.

A sound workflow includes:

• defining the target, time frequency and forecast horizon;
• auditing timestamps, missing values, duplicate records and irregular intervals;
• visualising the series and relevant external variables;
• establishing a simple baseline;
• selecting candidate models;
• validating them using only information available at each forecast origin;
• inspecting residuals and prediction intervals;
• deploying, monitoring and periodically recalibrating the model.

The distinction between a forecast and an explanation is important. A model may forecast accurately without identifying causal relationships. If management plans to change prices or staffing based on a model, causal analysis may be needed in addition to forecasting.

Baselines: The Necessary Point of Comparison

A naïve forecast uses the latest observation:

\[\hat y_{t+h|t}=y_t.\]

It is often surprisingly competitive for short horizons. A drift method extrapolates the average historical change, while a seasonal-naïve method uses the observation from the corresponding previous season:

\[\hat y_{t+h|t}=y_{t+h-m}.\]

For example, a monthly forecast may use the value from the same month in the previous year. Baselines reveal whether a sophisticated model adds practical value. They also provide a benchmark for communication: a model that cannot outperform a seasonal-naïve forecast may not justify its complexity.

Exponential Smoothing and State-Based Forecasts

Exponential smoothing assigns greater weight to recent observations while retaining a gradually diminishing influence from older observations. Simple exponential smoothing is suitable for a series with no systematic trend or seasonality. Its level component is updated as

\[\ell_t=\alpha y_t+(1-\alpha)\ell_{t-1},\]

where \(0<\alpha<1\). A high value of \(\alpha\) makes the forecast responsive but potentially noisy.

Holt’s method adds a trend component. Holt–Winters methods add seasonality and may use additive or multiplicative seasonal effects. Additive seasonality assumes a roughly constant seasonal magnitude; multiplicative seasonality assumes that the magnitude changes with the level.

These models are often strong operational choices because they are interpretable, computationally efficient and effective for many univariate business series. Their limitations include difficulty handling abrupt structural breaks, complex external drivers and multiple irregular seasonalities. Modern implementations may optimise smoothing parameters and estimate uncertainty, but model assumptions still require examination.

ARIMA and Seasonal ARIMA

The ARIMA framework models a series through autoregression, differencing and moving-average error terms. An ARIMA(p,d,q) model contains:

• p: the order of the autoregressive component;
• d: the number of non-seasonal differences;
• q: the order of the moving-average component.

An autoregressive model predicts a value from previous values, for example:

\[y_t=c+\phi_1y_{t-1}+\cdots+\phi_py_{t-p}+\varepsilon_t.\]

A moving-average component models the current value through previous forecast errors. In ARIMA, the moving-average terminology refers to errors, not to a simple rolling average.

When a series contains seasonality, SARIMA extends the model using seasonal orders, commonly written as SARIMA(p,d,q)(P,D,Q)m, where \(m\) is the seasonal period. An ARIMAX or dynamic-regression model incorporates external predictors, such as price, promotions or weather. The forecaster must know whether future predictor values are available. If they must themselves be forecast, their uncertainty should be propagated or the predictors should be scenario-based.

Model identification can use plots, information criteria and domain knowledge. AIC or BIC can assist comparison among fitted models, but the decisive test is performance on temporally held-out data. After fitting, residuals should approximate uncorrelated noise. A significant residual autocorrelation pattern indicates that the model has not captured all predictable temporal structure.

Machine Learning for Sequential Prediction

Machine-learning models can forecast time series when relevant predictors are available or when nonlinear relationships are important. The series must first be represented as supervised-learning data. For a one-step model, features might include lagged values \(y_{t-1},y_{t-7},y_{t-28}\), rolling means, rolling standard deviations, calendar indicators and known future variables.

Rolling features must be calculated using only observations available before the prediction time. A rolling mean that includes \(y_t\) when predicting \(y_t\) is a leakage error. Similarly, a centred moving average uses future values and should not be used as a predictive feature.

Common approaches include regularised linear regression, random forests, gradient-boosted trees and recurrent or attention-based neural networks. Tree-based models can capture nonlinear interactions and work well with calendar and business variables. Neural networks may be valuable with very large, high-frequency or multivariate data, but they require careful tuning, substantial data and rigorous validation.

Two strategies are common for multi-step forecasting. In recursive forecasting, a one-step model generates a forecast, feeds it back as a lag and predicts the next step. Errors can accumulate. In direct forecasting, a separate model predicts each horizon. This may reduce error propagation but requires more models. Multi-output approaches predict several horizons jointly. The appropriate strategy depends on horizon, data volume and operational requirements.

Validation Without Temporal Leakage

Random train-test splitting is generally inappropriate for forecasting. It permits the model to learn from future regimes and can make lagged features appear more informative than they would be in deployment. A basic holdout reserves the latest block of observations as a test period. A stronger approach is rolling-origin evaluation.

In an expanding-window design, the model is trained on an initial period, predicts the next horizon, then the training window expands and the process repeats. In a sliding-window design, the training window moves forward while retaining a fixed length. Sliding windows can adapt more quickly to concept drift but discard older information.

Validation must reproduce the intended deployment situation. If forecasts are generated once per month for the next three months, validation should mimic that origin and horizon. Hyperparameters must be selected using training and validation periods; the final test period should remain untouched until model selection is complete.

Leakage can arise from several sources:

• random splitting of ordered observations;
• scaling or imputation fitted on the complete dataset;
• rolling statistics that include future values;
• revisions to data that were unavailable at prediction time;
• using future promotional plans that were later cancelled;
• target-derived variables constructed after the forecast timestamp.

A leakage audit should document the information timestamp for every feature and transformation.

Measuring Forecast Accuracy and Uncertainty

Mean absolute error (MAE) is

\[MAE=\frac{1}{n}\sum_{t=1}^{n}|y_t-\hat y_t|.\]

It is easy to interpret in the target’s units. Root mean squared error (RMSE) penalises large errors more heavily. Mean absolute percentage error (MAPE) is problematic when actual values are zero or near zero. Weighted or symmetric alternatives may be preferable. Scaled measures such as MASE compare errors with a naïve benchmark and support comparisons across series.

The metric should reflect the decision cost. Under-forecasting demand may be more expensive than over-forecasting if stockouts are severe. Quantile forecasts provide asymmetric prediction intervals, such as the 90th-percentile demand needed for safety stock. Forecast intervals should be assessed for calibration: approximately the stated proportion of actual outcomes should fall within the interval over repeated evaluations.

A forecast can be accurate on average while being systematically biased during promotions, holidays or demand surges. Therefore, evaluate accuracy by horizon, segment, season and operational regime, not only through one aggregate score.

Illustrative Case Study: Forecasting Daily Pharmacy Demand

Background

A regional pharmacy chain wishes to forecast daily demand for a temperature-sensitive medicine at each branch. Excess inventory creates disposal costs, while shortages can delay treatment. The chain has two years of daily sales, branch identifiers, stock availability, weekday indicators, public-holiday information and recorded promotional activity.

Situation and Problem

Several branches show weekly patterns and occasional demand spikes during respiratory illness periods. A data scientist initially proposes a gradient-boosted model using random 80:20 splitting. The reported MAE is low, but managers observe poor performance when the model is used prospectively.

Analysis

Investigation reveals that the random split placed later observations in the training set. Rolling demand features were also calculated over the full data table, allowing information from the forecast day to enter the predictors. Moreover, sales during stockouts were treated as true demand, although observed sales were constrained by availability.

Plots show a stable weekly pattern, a gradual trend and several structural changes. A seasonal-naïve baseline using the same weekday from the previous week performs reasonably well. Residual analysis indicates that a simple model misses holiday effects but captures much of the routine variation.

Approach and Solution

The team defines a seven-day forecast horizon and uses rolling-origin validation. The candidate models are seasonal naïve, Holt–Winters, SARIMA with calendar regressors and a gradient-boosted model using lagged demand, trailing statistics, weekday, holiday status and known promotional schedules. Every feature is generated as it would have been available at the forecast origin. Scaling and imputation are fitted separately within each training fold.

The boosted model performs best during ordinary weeks, while SARIMA provides more stable forecasts during sparse periods. A forecast combination is selected: the machine-learning model supplies the central forecast, while the statistical model acts as a comparator and fallback. Quantile forecasts are used for replenishment decisions, and stockout days are flagged for separate treatment rather than interpreted as low demand.

Outcome

The revised evaluation produces a more realistic estimate of performance. The organisation adopts weekly monitoring by branch and horizon, with automatic alerts when error or bias exceeds a threshold. Managers retain the ability to override forecasts when new clinical guidance or supply disruption is known but not represented in historical data.

Learning Points

The case demonstrates that validation design can matter more than algorithm choice. It also shows why observed sales may not equal unconstrained demand, why known future information must be distinguished from unknown future information, and why model governance includes operational overrides and monitoring.

Challenges, Risks and Professional Judgement

Time series can contain missing intervals, changing measurement systems, outliers, interventions and structural breaks. An unusual observation may be an error, a legitimate shock or evidence that the underlying process has changed. Automatic outlier removal can destroy precisely the events that a forecasting system must learn to handle.

Non-stationarity may arise from inflation, market growth, technology adoption or policy change. A model trained on a historical regime may fail after a product launch or supply disruption. Concept drift should be monitored through rolling error, residual behaviour and feature-distribution changes.

Forecast uncertainty is often understated because models treat estimated parameters, future predictors and structural change too optimistically. Decision-makers should receive ranges and scenarios rather than a single apparently precise number. Forecasts should not be presented as guarantees.

Ethically, forecasting systems can influence access to healthcare, credit, employment and public services. Poorly calibrated forecasts may allocate resources inequitably across regions or populations. Governance should include documentation of data sources, forecast horizon, known limitations, performance by relevant subgroup, access controls and an accountable human decision process. Sensitive personal data should not be included merely because it improves predictive accuracy.

Technology and Emerging Practice

Modern forecasting platforms support automated pipelines, feature stores, experiment tracking, model registries and continuous monitoring. Cloud systems can generate forecasts for thousands of related series, while hierarchical forecasting reconciles forecasts across levels such as product, branch and total company. Probabilistic forecasting, quantile regression and distributional neural networks are increasingly used when decisions depend on risk rather than only on a point estimate.

Large pre-trained time-series models and transformer-based architectures are emerging for multivariate and cross-series forecasting. Their usefulness depends on data quality, computational cost, transferability and interpretability. Automation can accelerate model comparison, but automated selection does not remove the need for leakage audits, domain review and governance.

Best Practices for Applied Forecasting

A professional forecasting system should:

1. Begin with a clearly defined decision, horizon and data-availability rule.

2. Plot the series and investigate calendar, operational and structural effects.

3. Always report naïve and seasonal-naïve baselines.

4. Use rolling-origin validation that mirrors deployment.

5. Generate lagged and rolling features only from past information.

6. Compare accuracy by horizon, segment and regime.

7. Inspect residual autocorrelation, bias and interval calibration.

8. Prefer the simplest model that meets the decision requirement.

9. Monitor drift, data quality and forecast error after deployment.

10. Document assumptions, overrides, uncertainty and accountability.

Chapter Summary

Time-series forecasting uses temporally ordered information to estimate future values. Trend, seasonality, cycles, noise and structural change must be distinguished before modelling. Autocorrelation describes dependence across lags, while stationarity concerns the stability of a series’ statistical properties. Naïve and seasonal-naïve methods establish essential benchmarks. Exponential smoothing provides flexible level, trend and seasonal models; ARIMA and SARIMA represent autoregressive, differencing, moving-average and seasonal structures. Machine-learning models can use lagged, rolling, calendar and external features, but these features must be created without temporal leakage. Rolling-origin validation, decision-relevant metrics and uncertainty estimates are central to trustworthy comparison. Deployment additionally requires monitoring, governance, human judgement and awareness that forecasts support decisions but do not establish causality.

Key Terms

Stationarity: A condition in which key statistical properties of a series remain stable over time.

Trend: A persistent long-term increase or decrease in a time series.

Seasonality: A repeating pattern associated with a known time period.

Cycle: A longer-term fluctuation whose duration is not necessarily fixed.

Autocorrelation: Correlation between observations separated by a specified lag.

Lag: The number of time steps separating two observations.

ACF: The autocorrelation function across multiple lags.

PACF: The partial autocorrelation at each lag after controlling for shorter lags.

Differencing: Subtracting a previous observation to reduce non-stationarity.

Naïve forecast: A forecast that uses the latest observed value.

Seasonal-naïve forecast: A forecast that repeats the observation from the corresponding previous season.

Exponential smoothing: A method that weights recent observations more heavily than older observations.

Holt–Winters method: Exponential smoothing with level, trend and seasonal components.

ARIMA: A model combining autoregression, differencing and moving-average errors.

SARIMA: ARIMA extended with seasonal components.

Forecast origin: The time at which information is available and a forecast is generated.

Temporal leakage: Use of information in model development that would not be available at forecast time.

Rolling-origin validation: Repeated evaluation using successive historical forecast origins.

Concept drift: Change in the relationship between predictors and the target over time.

Prediction interval: A range intended to contain a future observation at a stated probability level.

Review Questions

Short Answer Questions

1. Distinguish trend from seasonality in a time series.

2. What does a lag-one autocorrelation measure?

3. Why is a seasonal-naïve forecast a useful benchmark?

4. Define temporal leakage and give one example.

5. Why may MAPE be inappropriate when actual values are zero?

Long Answer Questions

1. Explain stationarity, differencing and the role of ACF and PACF in ARIMA modelling.

2. Compare simple exponential smoothing, Holt’s method and Holt–Winters methods.

3. Describe how ARIMA and SARIMA models represent temporal dependence.

4. Explain how a time series can be converted into a machine-learning dataset without leakage.

5. Discuss the design of a rolling-origin forecasting evaluation, including training, validation and testing.

Analytical / Application Questions

1. A retailer has strong weekly seasonality and increasing demand. Propose a baseline, two candidate models and a validation design. Justify each choice.

2. A model has low overall MAE but consistently under-forecasts during holidays. Analyse possible causes and recommend corrective actions.

3. A hospital forecasts admissions using weather forecasts that are themselves uncertain. Explain how this uncertainty should influence model evaluation and operational decisions.

22. Multiple Choice Questions with Answers

Question 1

Which validation design best preserves the information available at the time of forecasting?

A. Randomly shuffle all observations before splitting

B. Use rolling-origin evaluation with successive historical forecast origins

C. Train on the test period and evaluate on the earliest observations

D. Use only observations with the largest target values

Correct Answer: B. Use rolling-origin evaluation with successive historical forecast origins

Question 2

A monthly series has a strong annual pattern. Which baseline is most appropriate for comparison?

A. The global mean of the series

B. The latest observation for every future month

C. The observation from the same month in the previous year

D. A randomly selected historical observation

Correct Answer: C. The observation from the same month in the previous year

Question 3

Which feature construction creates temporal leakage when predicting y_t?

A. The value y_(t-7)

B. A rolling mean calculated from observations through t-1

C. A weekday indicator known before t

D. A centred rolling mean that includes y_(t+1)

Correct Answer: D. A centred rolling mean that includes y_(t+1)

Question 4

In ARIMA(p,d,q), what does d represent?

A. The number of seasonal periods

B. The order of differencing

C. The number of external predictors

D. The forecast horizon

Correct Answer: B. The order of differencing

Question 5

Which method is designed to model level, trend and seasonality through recursive smoothing?

A. Holt–Winters exponential smoothing

B. Ordinary random splitting

C. Principal component analysis

D. A confusion matrix

Correct Answer: A. Holt–Winters exponential smoothing

Question 6

Why can a random train-test split produce an overly optimistic forecasting result?

A. It always removes seasonality

B. It prevents the model from using lagged values

C. It may allow future observations or regimes to influence training

D. It guarantees that the test set is too small

Correct Answer: C. It may allow future observations or regimes to influence training

Question 7

A forecast model performs well on ordinary days but fails during promotions. What is the most appropriate first response?

A. Delete all promotional observations

B. Evaluate performance by regime and include known promotional information

C. Report only the ordinary-day MAE

D. Increase model complexity without further analysis

Correct Answer: B. Evaluate performance by regime and include known promotional information

Question 8

Which statement about a prediction interval is correct?

A. It is a guaranteed range for every future value

B. It communicates uncertainty around a forecast at a stated coverage level

C. It is identical to a confidence interval for a model coefficient

D. It eliminates the need for monitoring

Correct Answer: B. It communicates uncertainty around a forecast at a stated coverage level

Question 9

What is a principal risk of recursive multi-step forecasting?

A. The model cannot use any lagged variables

B. Errors from earlier predictions can propagate into later forecasts

C. It always produces unbiased forecasts

D. It requires no validation data

Correct Answer: B. Errors from earlier predictions can propagate into later forecasts

Question 10

Why should a forecasting project begin with the decision and horizon being supported?

A. The decision determines the relevant frequency, error cost and evaluation design

B. The horizon determines whether data cleaning is unnecessary

C. The decision guarantees that a neural network is optimal

D. The horizon removes the need for a baseline

Correct Answer: A. The decision determines the relevant frequency, error cost and evaluation design