Time-Series Forecasting

Introduction and context

Time-series forecasting estimates future values from observations recorded in time order. Examples include weekly sales, hourly electricity demand, monthly inflation, daily website traffic, and quarterly revenue. Unlike ordinary cross-sectional prediction, time-series analysis must respect temporal ordering: information available at time t may be used to predict the future, but future observations must not influence a past forecast.

The business value of forecasting is not limited to producing a single number. Forecasts support inventory decisions, staffing, budgeting, capacity planning, cash management, maintenance scheduling, and risk control. A useful forecasting system also communicates uncertainty, updates as new data arrive, and is evaluated under conditions resembling deployment.

Learning objectives

By the end of this chapter, you should be able to:

• distinguish trend, seasonality, cycles, irregular variation, and structural breaks;
• explain stationarity and diagnose autocorrelation;
• construct naïve, seasonal-naïve, moving-average, and exponential-smoothing forecasts;
• describe and apply ARIMA models and interpret their parameters;
• engineer time-aware features for machine-learning forecasts;
• evaluate forecasts using rolling-origin, multi-step validation;
• select a model while considering uncertainty, operational costs, and governance.

1. The structure of a time series

Let y_t denote the observed value at time t, where the time interval is fixed and known. A useful conceptual representation is

where T_t is the trend, S_t is seasonal variation, C_t is cyclical variation, and I_t is irregular or unexplained variation. This additive decomposition is appropriate when seasonal fluctuations have roughly constant absolute size. A multiplicative form is often more suitable when seasonal amplitude grows with the level:

The multiplicative model requires meaningful positive values. Taking logarithms converts it into an additive form, log(y_t) = log(T_t) + log(S_t) + log(C_t) + log(I_t), when a log transformation is defensible.

Trend is a persistent long-term direction. It may be approximately linear, nonlinear, or interrupted by a structural change. Seasonality is a recurring pattern with a known and stable period, such as seven days for weekday effects or twelve months for annual seasonality. Cycles are fluctuations with less predictable duration, often related to economic or industry conditions. A business cycle is not the same as seasonality because its period is not fixed. Irregular variation includes shocks, measurement noise, and effects not represented by the model.

A time series may also contain missing observations, unequal time intervals, outliers, calendar effects, changing variance, and interventions such as promotions or policy changes. These features should be investigated before model fitting.

2. Preparing and exploring time-series data

A professional workflow begins with data auditing:

1. Confirm the time zone, timestamp definition, frequency, and measurement units.
2. Sort records chronologically and identify duplicates.
3. Check gaps, missing values, censored observations, and revisions.
4. Distinguish a genuine zero from a missing or unavailable value.
5. Record interventions, holidays, promotions, price changes, and policy events.
6. Plot the series and relevant subseries, such as demand by weekday or month.

Aggregation can alter the apparent dynamics. For example, aggregating hourly demand to daily demand may reduce noise but conceal intraday seasonality. The forecasting frequency should reflect the decision being supported and the time required to act.

Stationarity

A weakly stationary series has a constant mean, constant variance, and autocovariance that depends only on the lag, not on calendar time. If γ_k is the autocovariance at lag k, stationarity requires

for all relevant t, where μ and σ² are constant. Stationarity does not mean that every observation is identical; it means that the probabilistic structure is stable.

Many forecasting models work best when the series, or a transformed version of it, is stationary. Trend and changing seasonal means violate this condition. Common remedies include differencing, seasonal differencing, logarithms, Box–Cox transformations, detrending, or explicitly modelling trend and seasonality.

The first difference is

and a seasonal difference with period m is

Differencing removes persistent level changes but can also remove meaningful long-run information or create overdifferencing. The order of differencing should therefore be kept as low as possible.

Visual evidence is important: a plot with a changing level or variance suggests non-stationarity. The autocorrelation function (ACF), unit-root tests such as the augmented Dickey–Fuller test, and complementary tests such as KPSS can provide evidence, but none should replace substantive understanding. Test results are sensitive to sample size, deterministic terms, breaks, and model specification.

Autocorrelation

Autocorrelation measures linear dependence between observations separated by a lag. The sample autocorrelation at lag k is commonly estimated as

where n is the number of observations and ȳ is the sample mean. It is dimensionless and lies between −1 and 1. Positive autocorrelation means high values tend to follow high values; negative autocorrelation indicates alternation or reversal.

The partial autocorrelation function (PACF) measures the association at lag k after controlling for intermediate lags. ACF and PACF plots help identify candidate autoregressive and moving-average orders, but they are diagnostic tools rather than automatic model selectors. ACF patterns can also be produced by trend, seasonality, or an omitted variable.

3. Baselines and decomposition

A baseline is a simple, transparent benchmark. No complex model should be accepted merely because it produces a forecast; it should improve on an appropriate baseline.

The naïve forecast is

for horizon h ≥ 1. It is often difficult to beat for a highly persistent series. The seasonal-naïve forecast is

where m is the seasonal period. For monthly data with annual seasonality, m = 12.

A drift forecast extends the average change observed between the first and last training values:

This formula assumes equally spaced observations and is a simple trend benchmark, not a causal law.

A moving average forecast over a window of width q is

It smooths noise but responds slowly to turning points. Weighted moving averages assign larger weights to more recent observations; the weights must be non-negative and sum to one.

Decomposition methods estimate components separately. Classical decomposition may use centered moving averages for trend-cycle estimation and seasonal indices. STL decomposition uses locally weighted smoothing and is more flexible for changing trends and robust treatment of outliers. Decomposition is valuable for interpretation and diagnostics, but the components should not automatically be treated as causal entities.

4. Exponential smoothing

Exponential smoothing gives more recent observations greater weight. Simple exponential smoothing is appropriate for a series with no systematic trend or seasonality:

where ℓ_t is the level and 0 < α < 1 is the smoothing parameter. The forecast for every future horizon is ŷ_t+h|t = ℓ_t. A larger α reacts more quickly to new observations but may be more sensitive to noise.

Holt’s method adds a trend component. One common additive formulation is

where b_t is the estimated trend and 0 < β < 1. Damped-trend methods reduce the trend contribution at long horizons and are often more realistic when indefinite growth is implausible.

Holt–Winters methods add seasonality. In an additive seasonal model, the forecast is conceptually formed from a level, trend, and seasonal component; a multiplicative version scales the forecast by a seasonal index. Software implementations differ in initialization and parameterisation, so the chosen formulation must be documented. Exponential-smoothing parameters are commonly estimated by minimising a training loss, often one-step-ahead squared errors, subject to valid parameter constraints.

Worked example: seasonal naïve forecasting

A retailer records monthly sales. Sales in June were 1,240 units, and sales in June of the previous year were 1,100 units. For a July forecast using a 12-month seasonal-naïve method, the relevant observation is July of the previous year, not June. If that value was 980 units, then

ŷ_July|June = y_{July, previous year} = 980 units.

The example illustrates two principles: seasonal-naïve forecasts preserve the prior seasonal position, and the forecast must use only information available at the forecast origin.

5. ARIMA models

An autoregressive model uses previous values. An AR(p) model is

where p is the autoregressive order, c is an intercept, φ_i are coefficients, and ε_t is an innovation with mean approximately zero and no predictable serial correlation. Stationarity imposes restrictions on the coefficients; for AR(1), y_t = c + φ₁y_t−1 + ε_t, stationarity requires |φ₁| < 1.

A moving-average model uses previous shocks:

An ARIMA(p,d,q) model applies d ordinary differences before fitting an ARMA(p,q) model. In operator notation, with the lag operator B y_t = y_t−1,

where φ(B) and θ(B) are autoregressive and moving-average polynomials. This compact notation is useful, but the model’s assumptions remain: stable dynamics after differencing, appropriately specified errors, and no leakage from future data.

Seasonal ARIMA, written SARIMA(p,d,q)(P,D,Q)_m, adds seasonal autoregressive, differencing, and moving-average terms with period m. It is useful for recurring patterns such as monthly or weekly seasonality.

Model development commonly follows these steps:

1. Plot and transform the data where justified.
2. Difference only as needed to obtain a plausible stationary series.
3. Use ACF, PACF, domain knowledge, and information criteria to propose orders.
4. Estimate parameters on the training data.
5. Inspect residuals.
6. Compare out-of-sample forecasts with baselines.

A good residual series should have little remaining autocorrelation, approximately stable variance, and no obvious structure. A residual ACF that remains large indicates that the model has not captured temporal dependence. Information criteria such as AIC reward fit while penalising parameter count, but an information criterion is not a substitute for rolling-origin forecast evaluation.

6. Machine-learning approaches

Machine-learning models can forecast time series when relationships are nonlinear, many external predictors are available, or interactions are important. They do not automatically understand time. Time must be represented through features such as:

• lags: y_t−1, y_t−7, and y_t−12;
• rolling means, medians, standard deviations, minima, and maxima;
• calendar variables such as weekday, month, holiday, and promotion status;
• elapsed time and trend terms;
• external regressors such as price, weather, advertising, or economic indicators;
• cyclical encodings, for example sine and cosine of a calendar position.

For a seasonal position p within a period m, a cyclical encoding can be written as

These two variables preserve the fact that the end and beginning of a cycle are close. A regression forecast might be

where coefficients quantify conditional associations under the model assumptions. Tree ensembles can capture nonlinearities but may extrapolate poorly beyond the range of training data. Neural sequence models can represent complex dependencies, yet they require substantial data, careful regularisation, and strong baselines.

The most serious machine-learning error is leakage. A rolling mean for predicting time t must be calculated using observations available before t, not including y_t or later values. Feature transformations, imputation, scaling, and hyperparameter selection should be fitted within each training window when they use data-dependent information.

Multi-step forecasts can be generated recursively, by feeding predictions back as future lags; directly, by fitting a separate model for each horizon; or through a multi-output model. Recursive methods accumulate error, while direct methods require more models. The choice depends on horizon, data volume, and operational constraints.

7. Evaluation and rolling-horizon performance

A random train-test split is generally inappropriate because it permits temporal leakage and produces an unrealistic information set. Use a chronological split: earlier observations for training and later observations for validation or testing.

For forecast errors e_t = y_t − ŷ_t, common metrics include:

MAE has the same units as the target and treats errors linearly. RMSE also has target units but penalises large errors more heavily. MAPE is scale-free but is undefined when y_t = 0 and unstable near zero. Weighted or scaled alternatives, such as WAPE or MASE, may be more appropriate. MASE scales absolute errors by the in-sample naïve forecast error; lower values indicate improvement over that benchmark, although the exact scaling convention must be stated.

Prediction intervals communicate uncertainty. A point forecast alone can encourage false precision. Intervals should be assessed for empirical coverage and sharpness; a nominal 80% interval should contain approximately 80% of future observations over many comparable cases, subject to sampling variation.

Rolling-origin evaluation

Suppose data are available from times 1 through n. Choose an initial training end T and a forecast horizon H. Train on 1,…,T, forecast the next H periods, record errors, then move the origin forward and repeat. In expanding-window evaluation, all earlier observations remain in the training set. In a sliding-window design, only the most recent W observations are used. Sliding windows can adapt to regime changes but discard older information.

For each origin, evaluate every operational horizon separately. For example, a model may be excellent one week ahead but weak four weeks ahead. Compare models on identical origins, horizons, transformations, and missing-data rules. Statistical tests such as the Diebold–Mariano test can compare predictive accuracy, but practical significance, costs, and multiple comparisons also matter.

8. Illustrative case: forecasting regional pharmacy demand

Background. A pharmacy chain must forecast weekly demand for a high-volume medicine across 40 stores. Stockouts are costly, but over-ordering creates expiry risk. Demand shows a gradual upward trend, strong winter seasonality, occasional promotion spikes, and a supply disruption in one year.

Problem. The operations team initially uses the last week’s demand. This produces unstable orders after promotions and systematically underestimates winter demand. Management wants a four-week forecast updated every week.

Analysis. Analysts first reconcile store calendars, identify stockout weeks, and mark promotions and the supply disruption. They plot aggregate and store-level demand, examine the 52-week seasonal lag, and compare variance before and after a logarithmic transformation. A seasonal-naïve model is selected as the mandatory benchmark. Candidate models are Holt–Winters, SARIMA with annual weekly seasonality, and a gradient-boosted tree using lagged demand, rolling features, promotion flags, holiday indicators, and weather forecasts.

Approach. The latest 20% of weeks is reserved as a final test period. Within the earlier data, rolling-origin validation uses an expanding window and four forecast horizons. Features are recomputed at each origin. Weeks affected by stockouts are not treated as ordinary low demand; where possible, latent demand is estimated with a documented inventory-adjustment rule. The model is selected using horizon-weighted MAE, with higher weight on the first two weeks because replenishment decisions are most sensitive there. Prediction intervals are produced for safety-stock planning.

Outcome. The machine-learning model performs best at one and two weeks, while SARIMA is more competitive at four weeks. The chain adopts a forecast combination: the short-horizon machine-learning forecast is blended with the seasonal-naïve forecast, and the longer-horizon forecast uses a seasonal statistical model. Forecasts are monitored by store, horizon, promotion status, bias, and interval coverage. The system includes a human override with a reason code rather than allowing unexplained manual changes.

Learning points. The strongest solution was not selected from a single holdout score. It depended on correct treatment of stockouts, realistic rolling evaluation, horizon-specific performance, interpretable benchmarks, and a decision-aware loss function. Forecasting demand is distinct from forecasting observed sales when inventory constraints censor demand.

9. Challenges, governance, and professional practice

Forecasts are conditional statements, not certainties. Structural breaks caused by regulation, pandemics, technological change, or competitor entry can invalidate historical relationships. Seasonality may evolve, and data revisions may create apparent real-time information that was unavailable when a historical decision was made.

Governance should document the data source, forecast origin, frequency, transformations, model version, parameters, external variables, exclusions, and evaluation design. Monitor forecast bias, error by segment, drift in feature distributions, missing-data rates, and interval coverage. A model that is accurate on average may systematically underforecast a region or customer group.

Ethical concerns arise when forecasts allocate scarce resources, influence staffing, or affect vulnerable customers. Proxy variables can reproduce inequity. Automated forecasts should therefore support accountable decisions, include review procedures, and preserve an audit trail. Privacy principles require collecting only necessary data and controlling access to granular records.

Contemporary systems increasingly combine classical models with machine learning, probabilistic forecasting, hierarchical reconciliation, and automated monitoring. Hierarchical forecasting ensures that store forecasts aggregate consistently to regional and national totals. Forecast combinations often improve robustness because different models capture different aspects of uncertainty. Large language models may assist with documentation or scenario interpretation, but they should not be treated as autonomous numerical forecasting engines without reproducible data pipelines and validation.

Best-practice checklist

• Establish the decision, horizon, frequency, and cost of errors before choosing a model.
• Plot the data and investigate calendar effects, missingness, outliers, and interventions.
• Use naïve and seasonal-naïve benchmarks.
• Preserve chronological order and prevent leakage in every transformation.
• Validate with rolling origins and report performance by horizon and segment.
• Inspect residuals and prediction-interval coverage.
• Prefer the simplest model that meets the operational requirement.
• Monitor drift, bias, overrides, and realised outcomes after deployment.
• Refit and recalibrate according to evidence, not an arbitrary schedule.

Chapter Summary

Time-series forecasting models ordered observations while respecting temporal information. Trend represents persistent direction, seasonality recurs at a known period, cycles have less predictable timing, and irregular variation contains shocks and noise. Stationarity concerns stable probabilistic behaviour; differencing and transformations can help, but excessive differencing is harmful. ACF and PACF diagnose temporal dependence and guide, but do not determine, model orders.

Naïve and seasonal-naïve methods provide essential benchmarks. Moving averages and exponential smoothing offer transparent smoothing, while Holt–Winters methods represent trend and seasonality. ARIMA models represent autoregressive dependence, differencing, and moving-average shocks. Machine-learning methods can incorporate nonlinearities and external predictors, provided that lagged and rolling features are built without leakage.

Reliable evaluation uses chronological, rolling-origin validation and reports horizon-specific metrics such as MAE and RMSE, with careful treatment of zero values and scale. Prediction intervals, residual diagnostics, monitoring, governance, and decision costs are as important as point accuracy. In practice, robust forecasting often combines statistical and machine-learning models and treats data quality, structural change, and operational context as central analytical issues.

Key Terms

Stationarity: A condition in which mean, variance, and autocovariances are stable over time.

Trend: A persistent long-term direction in a series.

Seasonality: A recurring pattern with a known period.

Cycle: A fluctuation whose duration is not fixed or fully predictable.

Autocorrelation: Correlation between observations separated by a specified lag.

ACF: The autocorrelation function across multiple lags.

PACF: Partial autocorrelation after controlling for intermediate lags.

Lag: The number of time intervals separating two observations.

Differencing: Subtracting an earlier observation from a later one to reduce non-stationarity.

Naïve forecast: A forecast that carries the latest observation forward.

Seasonal-naïve forecast: A forecast that repeats the value from the corresponding prior season.

Exponential smoothing: A method assigning geometrically declining weights to older observations.

ARIMA: A model combining autoregression, differencing, and moving-average terms.

Innovation: A new, unpredictable shock or error in a time-series model.

Feature leakage: Use of information unavailable at the time a forecast would have been made.

Rolling-origin evaluation: Repeated forecasting from successive historical origins.

Forecast horizon: The distance into the future being predicted.

Prediction interval: An interval intended to contain a future observation with a stated probability.

Structural break: A change in the data-generating process.

Hierarchical forecasting: Forecasting related levels that must aggregate consistently.

Review Questions

Short Answer Questions

1. What distinguishes seasonality from a business cycle?
2. Why is a random train-test split generally unsuitable for time-series forecasting?
3. Define stationarity in terms of mean, variance, and autocovariance.
4. What information does the PACF provide?
5. State one advantage and one limitation of RMSE.

Long Answer Questions

1. Explain the additive and multiplicative decomposition models and identify circumstances in which each is appropriate.
2. Compare simple exponential smoothing, Holt’s method, and Holt–Winters methods.
3. Explain the roles of p, d, and q in an ARIMA model and describe a sensible model-development workflow.
4. Discuss how machine-learning models can forecast time series and explain at least three ways leakage can occur.
5. Design a rolling-origin evaluation plan for a monthly demand forecast with a six-month horizon.

Analytical / Application Questions

1. A monthly series rises over time and has increasingly large December peaks. Which transformation, decomposition approach, and benchmarks would you investigate first? Justify your choices.
2. A forecasting model has lower overall RMSE than its benchmark but systematically underforecasts during promotions. How should the analyst diagnose and respond to this result?
3. A retailer has forecasts for stores, regions, and the national total, but the independently generated forecasts do not add up. Explain the business risks and propose a reconciliation strategy.

Multiple Choice Questions with Answers

Question 1

Which feature most clearly distinguishes seasonality from a cycle?

A. Seasonality repeats at a known period

B. Seasonality is always caused by inflation

C. A cycle must repeat every seven days

D. A cycle cannot contain random variation

Correct Answer: A. Seasonality repeats at a known period

Question 2

For a seasonal-naïve forecast with monthly data and annual seasonality, which observation forecasts July?

A. June of the current year

B. July of the previous year

C. December of the previous year

D. The average of all months

Correct Answer: B. July of the previous year

Question 3

What does stationarity require in a weakly stationary series?

A. Every observation must equal the mean

B. The mean, variance, and autocovariance structure remain stable

C. The series must have a positive trend

D. All autocorrelations must be zero

Correct Answer: B. The mean, variance, and autocovariance structure remain stable

Question 4

Which metric is especially sensitive to large errors because it squares them before averaging?

A. MAE

B. MAPE

C. RMSE

D. Median error

Correct Answer: C. RMSE

Question 5

In simple exponential smoothing, what is the effect of increasing α?

A. The forecast responds more quickly to recent observations

B. The seasonal period becomes longer

C. All historical observations receive equal weight

D. The model necessarily becomes stationary

Correct Answer: A. The forecast responds more quickly to recent observations

Question 6

For an AR(1) process, which condition is associated with stationarity?

A. φ₁ = 0

B. |φ₁| < 1

C. |φ₁| > 1

D. φ₁ must equal 1

Correct Answer: B. |φ₁| < 1

Question 7

Which practice is an example of time-series leakage?

A. Using a lagged value available before the forecast origin

B. Using a holiday calendar known in advance

C. Including the current target value in a rolling mean used to predict that target

D. Evaluating forecasts on a later chronological period

Correct Answer: C. Including the current target value in a rolling mean used to predict that target

Question 8

Why is MAPE problematic when actual values can be zero?

A. It requires squared errors

B. It is undefined when an actual value is zero

C. It always has the target’s original units

D. It cannot compare two models

Correct Answer: B. It is undefined when an actual value is zero

Question 9

What is the main purpose of rolling-origin evaluation?

A. To imitate repeated historical forecasting using only information available at each origin

B. To randomise observations before training

C. To remove all seasonality automatically

D. To guarantee that the most complex model wins

Correct Answer: A. To imitate repeated historical forecasting using only information available at each origin

Question 10

A model’s residual ACF shows substantial correlation at several lags. What does this most strongly suggest?

A. The residuals contain remaining temporal structure

B. The model is necessarily unbiased

C. The forecast intervals are perfectly calibrated

D. The target variable has no trend

Correct Answer: A. The residuals contain remaining temporal structure