Time-Series Machine Learning and Forecasting

Introduction and context

Time-series forecasting estimates future values from observations ordered in time. It supports inventory replenishment, workforce scheduling, revenue planning, energy balancing, credit-risk monitoring, capacity management, and many other decisions. Unlike ordinary supervised learning, time-series modelling must respect temporal order. Information available after a forecast origin cannot be used to predict an earlier observation. This requirement affects data preparation, validation, feature engineering, model selection, and the interpretation of performance.

A time series may contain a systematic trend, recurring seasonal movements, serial dependence, calendar effects, interventions, structural breaks, and changing variance. A useful forecaster therefore combines statistical understanding with machine-learning discipline. The objective is not merely to minimize an error score; it is to produce forecasts that are timely, calibrated, explainable enough for the decision, and robust to plausible changes in the data-generating process.

Learning objectives

By the end of this chapter, a reader should be able to:

1. distinguish trend, seasonality, cycles, noise, autocorrelation, and structural change;
2. construct time-aware training, validation, and testing procedures;
3. create lag, rolling-window, calendar, and exogenous features without leakage;
4. explain and apply ARIMA-family models;
5. compare feature-based machine-learning forecasters with classical models;
6. select appropriate forecasting metrics and interpret uncertainty intervals;
7. design a practical forecasting workflow for managerial decision support; and
8. identify limitations, governance risks, and contemporary developments in forecasting.

1. The structure of a time series

Let y_t denote the observed value at time t, where observations are recorded at a consistent frequency. A useful conceptual decomposition is

where T_t is the trend, S_t is a seasonal component, C_t represents longer or irregular cycles, and I_t is the remaining irregular component. This additive form is appropriate when seasonal fluctuations have approximately constant absolute size. When variation grows with the level, a multiplicative formulation is often more appropriate:

The multiplicative model requires non-negative observations. A logarithmic transformation can convert multiplicative relationships into additive ones, although forecasts must then be carefully back-transformed because the mean of a log-transformed variable is not generally the logarithm of its mean.

Trend and seasonality

Trend is persistent movement in the series’ level or slope. Seasonality is a pattern that repeats at a known and usually fixed calendar frequency, such as 12 months, 7 days, or 24 hourly observations. A cycle may resemble seasonality but has a less fixed period, often responding to economic or business conditions. A one-time promotion is neither trend nor seasonality; it is an intervention or event.

Seasonal patterns should be defined by the data-generating context. For example, weekly demand may be seasonal because customer behaviour differs by weekday, while annual demand may be seasonal because of holidays. A model using a generic time index cannot automatically learn such patterns unless relevant calendar information is supplied.

Autocorrelation and stationarity

Autocorrelation measures dependence between observations separated by a lag. The lag-k autocorrelation is

where Cov is covariance and Var is variance. The expression is meaningful when the relevant moments are stable enough over time; for a weakly stationary process, the mean and variance are constant and covariance depends only on the lag, not on calendar time.

A high autocorrelation does not prove causation. It may result from trend, seasonality, omitted variables, or genuine persistence. Differencing is often used to remove a stochastic trend:

Seasonal differencing at period m is

Differencing should be used only as needed. Over-differencing can amplify noise and create artificial negative autocorrelation. Analysts commonly inspect plots, seasonal summaries, autocorrelation functions, and stationarity tests, while recognizing that formal tests have limited power and depend on assumptions.

ACF and PACF

The autocorrelation function (ACF) displays correlations at successive lags. The partial autocorrelation function (PACF) measures the association at a lag after accounting for shorter lags. In classical identification, an AR process often has a PACF that cuts off after its order, whereas an MA process often has an ACF that cuts off. In finite, noisy samples these patterns are only diagnostic clues, not definitive model-selection rules.

2. Forecasting horizons and information sets

A forecast made at time t for time t+h is written \hat{y}_t+h|t, where h is the horizon and the conditioning notation indicates that only information available through t is used. One-step forecasts and long-horizon forecasts have different error behaviour. Recursive forecasting predicts one step, feeds that prediction back as an input, and continues. Direct forecasting fits a separate model for each horizon. Multi-output approaches predict several horizons jointly.

The information set should include only variables known at the forecast origin. Future planned prices may be valid inputs if they are genuinely fixed or reliably available; realized future sales, future weather observations, and revised economic indicators are not valid unless the operational process supplies forecasts of them. This distinction is central to avoiding leakage.

3. Temporal validation and forecast evaluation

Randomly splitting a time series into training and test rows is usually invalid because it allows future patterns to influence training and destroys the deployment sequence. A basic holdout reserves the latest observations as a test period. A stronger design uses rolling-origin evaluation:

1. train on an initial historical window;
2. forecast the next period or horizon;
3. move the forecast origin forward;
4. optionally expand or roll the training window; and
5. aggregate errors across origins.

An expanding window retains all earlier observations. A rolling window retains only the most recent fixed number, which can be useful when relationships decay or structural change is substantial. Validation windows should resemble the real decision process, including the actual forecast horizon and update frequency.

Error metrics

For actual values y_i and forecasts \hat{y}_i, with n evaluated observations, mean absolute error is

MAE is expressed in the target’s units and gives linear weight to errors. Root mean squared error is

RMSE penalizes large errors more heavily and is therefore sensitive to outliers. Mean absolute percentage error is

MAPE is undefined when y_i = 0 and unstable near zero. Symmetric MAPE has its own scaling and interpretation problems. Weighted absolute percentage error, or business-weighted loss, may be preferable when volumes or costs differ across periods. Metric choice should reflect the decision: a stockout-sensitive retailer may assign a much higher cost to underforecasting than to overforecasting.

Always compare complex models with meaningful baselines, such as the naïve forecast \hat{y}_t+1|t = y_t or the seasonal naïve forecast \hat{y}_t+h|t = y_t+h−m, where m is the seasonal period. A model that does not beat an appropriate baseline is not operationally persuasive.

4. Feature-based forecasting

Feature-based forecasting transforms temporal structure into predictors for a supervised-learning algorithm. Typical features include:

For a one-step model, a lagged feature can be written as x_t = y_t−1. For horizon h, a direct model may use x_t,h = (y_t, y_t−1, …) to estimate y_t+h. Rolling features must use a closed historical window, for example the mean of the previous seven observations:

Including y_t in this feature would leak the value being forecast when the forecast is issued before t. The same issue arises when scaling, imputing, selecting variables, or encoding categories using the complete dataset. These transformations should be fitted within each training fold.

Cyclical calendar variables can be represented with sine and cosine terms. For a position [[MATH}q_t]] in a cycle of length [[MATH}P]], use

Together, these features place the end and beginning of a cycle near each other. One-hot encoding is often easier to interpret for a small number of calendar categories; cyclical encoding is compact and useful when continuity is meaningful.

Tree ensembles such as random forests and gradient-boosted trees handle nonlinear interactions among lags, prices, promotions, and calendar variables. Regularized linear models provide simpler and often more stable baselines. Neural networks can learn complex temporal representations, but they require sufficient data, careful validation, and strong operational justification.

5. ARIMA-family methods

An autoregressive model of order p, AR(p), is

where c is an intercept, φ_i are autoregressive coefficients, and ε_t is an error term, commonly assumed to have mean zero and finite variance. Stationarity imposes restrictions on the coefficients; for an AR(1), |φ₁| < 1 is sufficient for covariance stationarity.

A moving-average model of order q, MA(q), is

where μ is the process mean and θ_i are moving-average coefficients. The errors should be uncorrelated innovations under the model assumptions.

ARIMA(p,d,q) applies d non-seasonal differences before fitting an ARMA model. In operator notation, with the lag operator B y_t = y_t−1,

where φ(B) and θ(B) are autoregressive and moving-average polynomials. Seasonal ARIMA, SARIMA, adds seasonal autoregressive, differencing, and moving-average terms at period m, commonly written ARIMA(p,d,q)(P,D,Q_m).

ARIMAX or dynamic regression includes external predictors. A simplified form is

where x_t is a vector of exogenous variables, β is a coefficient vector, and n_t follows an ARIMA-type error process. Exogenous variables must be known or forecast at the time the target forecast is produced.

Model diagnostics include residual plots, residual ACF, distributional checks, and stability across rolling origins. A good residual series should contain little predictable temporal structure. Normal residuals are convenient for interval construction but are not required for point forecasts; heavy tails, changing variance, and outliers may require robust methods or transformations.

6. Uncertainty and probabilistic forecasts

A point forecast alone is incomplete for planning. A prediction interval describes uncertainty about a future outcome under a specified model and information set. Under a simple one-step model with conditionally normal error ε_t+1 ∼ N(0,σ²), an approximate 95% interval is

where SE is the standard error of the forecast error. For longer horizons, uncertainty generally increases because future innovations accumulate and model inputs may themselves be uncertain. If the forecast distribution is skewed or the sample is small, normal intervals may be poorly calibrated.

A prediction interval concerns an individual future observation; a confidence interval concerns uncertainty about an estimated parameter or mean. They are not interchangeable. Quantile regression, conformal prediction, bootstrap simulation, Bayesian posterior predictive distributions, and distributional neural networks provide alternative approaches. Calibration should be checked empirically: among forecasts labelled 90% intervals, approximately 90% of comparable future observations should lie within the intervals, subject to sampling variation and stable conditions.

Forecast uncertainty has several sources: irreducible innovation noise, parameter estimation, uncertain future regressors, model choice, and structural change. Reporting one narrow interval based only on residual variance can mislead decision makers. Scenario analysis is useful when future prices, promotions, weather, or policy are uncertain.

7. A professional forecasting workflow

A defensible workflow is iterative rather than purely algorithmic.

Define the decision. Specify the target, frequency, horizon, update schedule, cost of errors, and required granularity. Forecasting daily demand for replenishment is different from forecasting quarterly revenue for budgeting.

Audit the data. Check timestamps, duplicated records, missing intervals, time zones, revisions, aggregation rules, outliers, censoring, and changes in measurement. Distinguish a true zero from a missing observation or stockout.

Establish baselines. Use naïve, seasonal-naïve, moving-average, or simple exponential-smoothing benchmarks. Baselines expose whether sophistication adds value.

Explore temporal structure. Plot the series, seasonal profiles, rolling mean and variance, ACF/PACF, event markers, and relationships with candidate exogenous variables. Exploration must itself be separated from information used in final evaluation when it influences model decisions.

Build leakage-safe features and folds. Fit preprocessing on training data only. Ensure that every predictor is available at the forecast origin. For multi-step forecasts, reproduce the actual production mechanism for future lags and external inputs.

Compare diverse model classes. A candidate set may include seasonal naïve, ETS or exponential smoothing, ARIMA/SARIMA, regularized regression, gradient boosting, and a carefully justified neural model. Choose based on rolling-origin performance, stability, interpretability, latency, and maintenance cost—not a single score.

Diagnose and monitor. Examine residual dependence, bias by horizon and segment, interval coverage, drift, missing inputs, and business outcomes. Retraining frequency should respond to evidence of change rather than arbitrary fashion.

8. Illustrative case: regional grocery demand

Background. A fictional grocery chain forecasts weekly demand for a fresh product in 40 stores. Orders are placed six days before delivery. Demand is strongly seasonal around holidays, differs by weekday mix, and is affected by price discounts and promotions.

Problem. The existing process uses last week’s sales. It produces excess waste after promotions and stockouts during holiday peaks. Management needs a forecast for each store and a range suitable for order decisions.

Analysis. The analytics team discovers that observed sales during stockouts are censored demand: a low recorded sale does not necessarily mean low customer demand. They also find a 52-week pattern, promotion effects, store-level differences, and a gradual change after a competitor opens nearby. A random train-test split gives an overly optimistic result because later holiday behaviour appears in training rows for earlier periods.

Approach. The team creates a seasonal-naïve benchmark, then compares SARIMA, a regularized lag model, and gradient-boosted trees. Features include lagged sales, trailing four-week demand, week-of-year, holiday indicators, price, planned promotion, store attributes, and a competitor indicator. Every feature is generated as it would have been known six days before delivery. Rolling-origin validation uses the most recent 12 months as repeated test windows, with special reporting for holiday weeks. Quantile models produce lower and upper order-planning forecasts. Promotion plans are treated as known inputs, while weather is supplied through the weather forecast available on the order date rather than realized future weather.

Outcome. The boosted model has the lowest overall MAE, but SARIMA is more stable for stores with sparse history. A weighted ensemble is deployed: the boosted model handles high-volume stores, while seasonal naïve or SARIMA serves low-volume stores. The ordering system uses a calibrated upper quantile when the cost of a stockout exceeds the cost of waste. Performance is monitored by store, horizon, promotion status, and holiday status rather than only as one chain-wide average.

Learning points. The case demonstrates that data quality, censoring, temporal validation, segment-specific modelling, and decision-based loss functions can matter more than selecting the newest algorithm. It also shows why prediction intervals and operational inputs must be designed together with the point forecast.

9. Managerial, ethical, and governance considerations

Forecasts influence procurement, staffing, credit limits, pricing, and service access. Systematic bias can therefore create material harm. A demand model trained on stockout-constrained sales may under-serve locations already experiencing shortages. A workforce model may reproduce historical understaffing in particular communities. Governance should document data provenance, forecast ownership, model versions, assumptions, known failure conditions, and override authority.

Important controls include access restrictions for commercially sensitive forecasts, audit logs for manual overrides, monitoring for subgroup and geographic bias, and clear communication that intervals represent uncertainty rather than guarantees. Human review should be structured: overrides should record the reason and later be evaluated, rather than becoming untraceable intuition. Regulatory obligations vary by application, but privacy, consumer protection, employment law, financial controls, and record-retention requirements may all apply.

10. Contemporary developments and critical evaluation

Modern forecasting increasingly uses global models trained across many related series. They can share statistical strength across products, stores, or customers and can incorporate static and dynamic covariates. Temporal convolutional networks, recurrent networks, transformers, and foundation-style time-series models are active research areas. Their advantages are greatest when there are many related series, rich covariates, and enough history; they are not automatically superior for a short, stable series.

Hybrid and ensemble forecasting combine models with different inductive biases. Hierarchical forecasting reconciles forecasts across levels, such as product, store, region, and total, so that lower-level forecasts add consistently to higher-level forecasts. Intermittent-demand methods address sparse, irregular sales where ordinary metrics and models can behave poorly. Causal forecasting separates prediction from estimating the effect of an intervention; a model that predicts sales accurately need not identify the causal effect of a price change.

Research and practice should resist leaderboard thinking. A small improvement in average error may be irrelevant if it requires unavailable inputs, increases latency, reduces interpretability, or fails during a structural break. The appropriate model is the one that produces reliable decision value under realistic information, operational, and governance constraints.

Best-practice checklist

• Define the forecast origin and information set explicitly.
• Preserve time order in every evaluation step.
• Use at least one naïve or seasonal-naïve benchmark.
• Generate lags and rolling statistics without future leakage.
• Match validation horizons to the business decision.
• Report errors by horizon, segment, season, and important events.
• Treat external-variable forecasts as uncertain inputs.
• Examine residual dependence and forecast bias.
• Calibrate intervals and use asymmetric costs when appropriate.
• Monitor drift, data quality, model performance, and override patterns.
• Prefer a maintainable model with demonstrated decision value over unnecessary complexity.

Chapter Summary

Time-series forecasting differs from ordinary supervised learning because temporal order and information availability are fundamental. Series may contain trend, fixed seasonality, cycles, autocorrelation, interventions, changing variance, and structural breaks. Decomposition, differencing, ACF/PACF analysis, and domain knowledge help describe these features.

Temporal validation uses chronological holdouts or rolling-origin evaluation rather than random splits. Feature-based models use lagged values, rolling statistics, calendar variables, trends, and exogenous predictors, but every feature must be available at the forecast origin. ARIMA and SARIMA model dependence through autoregressive, differencing, moving-average, and seasonal components; machine-learning models capture nonlinear interactions and can learn across related series. Neither family is universally best.

Forecast evaluation should use decision-relevant metrics and credible baselines. Point forecasts should be complemented by prediction intervals, quantiles, scenarios, or other probabilistic outputs. A professional workflow integrates data auditing, leakage-safe modelling, diagnostics, monitoring, governance, and human decision processes. Contemporary methods—including ensembles, hierarchical reconciliation, global models, and neural architectures—are useful when their data requirements and operational benefits are demonstrated.

Key Terms

Autocorrelation: Dependence between observations separated by a specified time lag.

Baseline forecast: A simple benchmark, such as naïve or seasonal-naïve forecasting, used for comparison.

Cross-validation: Model evaluation using repeated train-validation procedures; in time series it must preserve temporal order.

Differencing: Subtracting a previous observation from the current observation to reduce trend or achieve stationarity.

Exogenous variable: A predictor external to the target series whose future value is known or forecast.

Forecast origin: The time at which a forecast is produced and the information set is defined.

Forecast horizon: The distance between the forecast origin and the period being predicted.

Leakage: Use of information unavailable at the forecast origin, producing unrealistically strong evaluation results.

MAE: Mean absolute error, the average absolute difference between actual and forecast values.

MAPE: Mean absolute percentage error; problematic when actual values are zero or near zero.

PACF: Partial autocorrelation function, measuring lag association after controlling for shorter lags.

Prediction interval: A range intended to contain a future observation with a stated probability.

Rolling-origin evaluation: Repeated forecasting from successive historical origins.

SARIMA: Seasonal autoregressive integrated moving-average model.

Seasonality: A pattern that repeats at a known calendar frequency.

Stationarity: A property in which key distributional features, such as mean and variance, are stable over time.

Structural break: A persistent change in the data-generating process or model parameters.

Trend: Persistent long-term movement in a time series’ level or slope.

Review Questions

Short Answer Questions

1. Why is a random train-test split generally inappropriate for time-series forecasting?
2. Distinguish trend, seasonality, and a one-time intervention.
3. What information does the ACF provide?
4. Why can MAPE be problematic for demand data?
5. What is forecast leakage? Give one example.

Long Answer Questions

1. Explain the components of a professional forecasting workflow from problem definition through monitoring.
2. Compare ARIMA-family models with feature-based gradient-boosting models, including assumptions, strengths, and limitations.
3. Explain rolling-origin validation and discuss how expanding and rolling windows differ.
4. Discuss the sources of forecast uncertainty and methods for producing probabilistic forecasts.
5. Explain how ethical and governance considerations affect the design and deployment of business forecasts.

Analytical / Application Questions

1. A retailer has weekly demand with a 52-week seasonal pattern, frequent promotions, and occasional stockouts. Design a leakage-safe validation and modelling strategy, including suitable baselines, features, and metrics.
2. A model has lower overall RMSE than its competitor but consistently underforecasts holiday demand. How should the analyst investigate and respond?
3. A company plans to use future price and weather as predictors. Explain what must be true for these variables to be valid inputs and propose a strategy when their future values are uncertain.

Multiple Choice Questions with Answers

Question 1

Which validation design best respects the temporal ordering of observations?

A. Randomly assigning every row to training or testing

B. Training on earlier observations and testing on later observations

C. Sorting observations by target value before splitting

D. Using only the observations with the smallest errors

Correct Answer: B. Training on earlier observations and testing on later observations

Question 2

Which feature is most likely to cause leakage when forecasting y_t before time t?

A. y_{t−1}

B. The weekday associated with t

C. A rolling mean computed from y_{t−7} through y_{t−1}

D. y_t itself

Correct Answer: D. y_t itself

Question 3

What is the principal limitation of MAPE?

A. It cannot be expressed as a percentage

B. It is undefined when actual values are zero

C. It always penalizes underforecasting more than overforecasting

D. It requires normally distributed errors

Correct Answer: B. It is undefined when actual values are zero

Question 4

In an additive decomposition, which component represents a repeating calendar pattern?

A. Trend

B. Seasonality

C. Innovation error

D. Parameter uncertainty

Correct Answer: B. Seasonality

Question 5

What does seasonal naïve forecasting use for a forecast at horizon h?

A. The overall sample mean

B. The immediately preceding observation only

C. The observation from the corresponding previous season

D. The largest historical observation

Correct Answer: C. The observation from the corresponding previous season

Question 6

For an AR(1) process, which condition is sufficient for covariance stationarity?

A. φ₁ = 0

B. φ₁ > 1

C. |φ₁| < 1

D. φ₁ < −1

Correct Answer: C. |φ₁| < 1

Question 7

Which statement correctly distinguishes a prediction interval from a confidence interval?

A. A prediction interval concerns a future observation

B. A prediction interval is always narrower

C. A confidence interval must contain every future observation

D. They are different names for the same quantity

Correct Answer: A. A prediction interval concerns a future observation

Question 8

Why might a rolling training window be preferred to an expanding window?

A. It guarantees zero forecast error

B. It can emphasize recent relationships after structural change

C. It eliminates the need for validation

D. It makes future regressors known

Correct Answer: B. It can emphasize recent relationships after structural change

Question 9

Which model family explicitly uses autoregressive, differencing, and moving-average components?

A. SARIMA

B. Random forest only

C. Principal component analysis

D. K-means clustering

Correct Answer: A. SARIMA

Question 10

If stockouts limit observed sales, what is a key modelling concern?

A. Observed sales may understate unconstrained customer demand

B. Stockouts prove the series has no seasonality

C. The target becomes automatically normally distributed

D. The data can be randomly shuffled without consequence

Correct Answer: A. Observed sales may understate unconstrained customer demand