ARIMA, Seasonal Models and Diagnostics

Introduction and context

Many business series—sales, website traffic, electricity demand, inventories and call-centre volumes—are recorded sequentially over time. Their observations are therefore not usually independent. Recent demand may depend on earlier demand, and recurring calendar effects may produce seasonal patterns. Predictive analytics must account for this dependence rather than applying methods designed for randomly sampled observations.

ARIMA and SARIMA models provide a systematic framework for modelling autocorrelation, trend and seasonality. The central workflow is:

1. inspect and prepare the time series;
2. stabilise its variance when necessary;
3. difference it to achieve stationarity;
4. use ACF and PACF patterns to propose models;
5. estimate and compare candidate models;
6. diagnose residuals; and
7. evaluate forecasts using rolling-origin validation.

These steps are complementary. ACF and PACF support model identification, but they do not replace business knowledge, estimation or validation.

Learning objectives

By the end of this chapter, you should be able to:

• explain stationarity and why it matters for ARIMA modelling;
• apply transformations and regular or seasonal differencing;
• interpret ACF and PACF patterns for tentative model identification;
• specify and interpret ARIMA and SARIMA models;
• distinguish estimation, model selection and forecast evaluation;
• perform residual diagnostics using autocorrelation tests and graphical evidence;
• design a rolling-origin forecast evaluation; and
• communicate the limitations and responsible use of time-series forecasts.

1. Stationarity, transformations and differencing

1.1 What stationarity means

A weakly stationary series has a constant mean, constant variance and autocovariance that depends only on the lag, not on calendar time. If y_t is the observation at time t, weak stationarity requires:

and

where μ is the constant mean, σ² is the constant variance, k is a lag and γ_k is the autocovariance at that lag. These conditions are assumptions about the data-generating process, not merely visual properties of a plot.

Stationarity is important because the historical relationship between observations is then more plausibly relevant to future observations. A trend or changing variance can produce misleading correlations and unstable forecasts.

1.2 Transformations

A transformation changes the scale of the observations, often to stabilise variance or make effects additive. Common choices include:

The logarithm requires strictly positive observations. If zeros occur, an analyst should not automatically add an arbitrary constant; the choice should be documented and assessed for sensitivity. Forecasts made on a transformed scale must be back-transformed. For logarithmic forecasts, simply exponentiating the mean forecast can underestimate the mean on the original scale when forecast errors are non-negligible. A bias adjustment may therefore be required.

1.3 Regular differencing

First differencing removes a stochastic level component:

where B is the backshift operator, defined by By_t = y_t−1. The second difference is:

Differencing should be used only as much as needed. Under-differencing leaves nonstationarity; over-differencing can create unnecessary negative autocorrelation, increase forecast variance and obscure the underlying process. In many practical series, d is 0 or 1; a value of 2 requires strong evidence.

1.4 Seasonal differencing

For a seasonal period s, seasonal differencing is:

For monthly data with annual seasonality, s = 12; for quarterly data with annual seasonality, s = 4. Regular and seasonal differencing can be combined:

where d is the order of regular differencing and D is the order of seasonal differencing. The operators commute, so their order does not alter the result.

Differencing reduces the number of usable observations. A model with excessive differencing may appear sophisticated while performing poorly out of sample. Use plots, domain knowledge and formal tests such as ADF or KPSS as supporting—not automatic—evidence.

2. Autocorrelation and partial autocorrelation

2.1 ACF

The sample autocorrelation at lag k measures linear association between observations separated by k periods:

Here n is the sample size and ȳ is the sample mean. The ACF includes both direct and indirect relationships. For example, a strong lag-2 correlation may be partly transmitted through lag 1.

2.2 PACF

The partial autocorrelation at lag k measures the association between y_t and y_t−k after controlling for lags 1 through k−1. It is especially useful for identifying autoregressive order. PACF values may be obtained from successive autoregressions or equivalent Yule–Walker calculations.

Approximate ACF/PACF significance bands are often shown as ±1.96/√n. They are useful heuristics under approximate white-noise assumptions, but multiple lags, non-normality and parameter estimation affect their reliability. A spike outside the band is not automatically evidence that a particular model is correct.

2.3 Identification patterns

“Cut off” means that correlations become approximately zero after a small number of lags. In finite samples, patterns are rarely exact. ACF/PACF should be computed after appropriate transformation and differencing, not blindly on the raw series.

3. ARIMA models

An ARIMA(p,d,q) model applies an ARMA model to the d-times differenced series. Using the backshift operator, a conventional form is:

where:

• φ(B) = 1 − φ₁B − … − φ_pB^p is the autoregressive polynomial;
• θ(B) = 1 + θ₁B + … + θ_qB^q is the moving-average polynomial;
• c is an intercept or drift-related constant, depending on specification;
• ε_t is an innovation or error at time t, commonly assumed to have mean zero, constant variance and no serial correlation; and
• p, d and q are AR order, differencing order and MA order.

Sign conventions for MA coefficients differ across software. Always read the software documentation before interpreting a coefficient. A stationary AR component requires roots of φ(B)=0 to lie outside the unit circle; invertibility of the MA component similarly requires roots of θ(B)=0 to lie outside the unit circle. These conditions support a stable and identifiable representation.

3.1 Special cases

An AR(1) model is:

For stationarity, |φ₁| < 1. An MA(1) model is:

An ARIMA(0,1,0) model is a random walk:

With drift, the differenced series has a non-zero mean, producing a linear expected forecast path. A random walk can be a strong benchmark even when it is not the final selected model.

3.2 Estimation and selection

Parameters are commonly estimated by maximum likelihood, conditional sum of squares or related numerical methods. Maximum likelihood compares the probability of the observed data under candidate parameter values, subject to initial-value treatment and distributional assumptions.

Model selection should combine:

• information criteria such as AIC or BIC;
• coefficient plausibility and stability;
• residual diagnostics;
• forecast accuracy on data not used for fitting; and
• operational interpretability.

AIC is commonly written:

and BIC as:

where ℓ is maximised log-likelihood, k is the number of estimated parameters and n is the effective sample size. Lower values are preferred among models fitted to the same response and comparable likelihood. BIC penalises complexity more strongly as n grows. Neither criterion guarantees superior forecast accuracy.

4. Seasonal ARIMA models

A SARIMA model extends ARIMA by representing both non-seasonal and seasonal dynamics. It is written SARIMA(p,d,q)(P,D,Q_s), with multiplicative form:

Here P, D and Q are seasonal AR order, seasonal differencing order and seasonal MA order; s is the seasonal period; and Φ(B^s) and Θ(B^s) are seasonal polynomials. For example:

Seasonal differencing is suggested by persistent ACF spikes at seasonal lags and an unstable seasonal level. After seasonal differencing, residual seasonal spikes may suggest seasonal AR or MA terms. A seasonal MA(1) can be represented by dependence on ε_t−s, while a seasonal AR(1) depends on y_t−s after the relevant differencing.

A practical identification sequence is:

1. plot the series and identify the data frequency;
2. assess whether variance rises with level;
3. transform if justified;
4. inspect seasonal plots and seasonal lags;
5. apply regular and/or seasonal differencing conservatively;
6. inspect ACF and PACF of the resulting series;
7. fit a small set of plausible non-seasonal and seasonal models; and
8. diagnose and validate them.

Do not use s = 12 merely because data are monthly: the business process must plausibly have an annual cycle.

5. Residual diagnostics

A fitted model is not adequate merely because its AIC is low. Let e_t = y_t − ŷ_t|t−1 denote a one-step-ahead residual, or the appropriate innovation residual produced by the estimation method. Good residuals should approximate white noise:

• mean near zero;
• stable variance;
• no meaningful autocorrelation; and
• no systematic structure by time, season or fitted value.

5.1 Graphical checks

A time plot of residuals can reveal level shifts, variance changes and outliers. A residual ACF can reveal remaining serial dependence. A histogram or Q–Q plot can assess non-normality. Normality is not required for unbiased point forecasts, but it matters for likelihood-based inference and prediction intervals, especially in small samples or with extreme observations.

Residuals should also be checked against fitted values and calendar periods. A seasonal pattern that remains in residuals indicates that the model has not captured all systematic structure.

5.2 Ljung–Box test

The Ljung–Box statistic at a selected maximum lag m is:

where r_k is the residual autocorrelation at lag k. Under the null hypothesis that residual autocorrelations through lag m are jointly zero, Q is approximately chi-squared with degrees of freedom adjusted for estimated model parameters. A small p-value suggests remaining serial correlation.

The test is not a proof of adequacy. Its result depends on the chosen lag, sample size and model complexity. A large p-value may reflect low power, and a very large sample may make trivial correlations statistically significant. Use the test with plots and business interpretation.

5.3 Outliers and interventions

An isolated shock, such as a promotion, strike or system outage, can distort parameter estimates and ACF/PACF patterns. Analysts should investigate whether it is:

• a recording error requiring correction;
• a one-time intervention requiring a dummy variable;
• a pulse that affects one period; or
• a permanent level or trend change.

Deleting observations without documentation is poor practice. Forecast intervals should communicate uncertainty associated with future shocks, which historical residuals may understate.

6. Rolling-origin forecast evaluation

A single train–test split can be misleading because results depend on the selected cutoff. Rolling-origin evaluation repeatedly mimics the forecasting task. At origin T_j, fit the model using observations up to T_j and forecast the next h periods. Then move the origin forward and repeat.

Two common designs are:

• expanding window: all observations up to the origin are retained;
• rolling window: only the most recent fixed number of observations is retained.

Expanding windows are appropriate when older data remain relevant. Rolling windows may be useful after structural change or when the process evolves.

For actual values y_t and forecasts ŷ_t, common measures include:

Here N is the number of evaluated forecasts. MAE has the units of the series and is easily interpreted. RMSE penalises large errors more heavily. MAPE is undefined when y_t = 0 and can be unstable near zero; it should not be used automatically. For intermittent or zero-heavy demand, scaled measures or MAE may be preferable.

Forecast comparisons should use the same forecast origins, horizon, target scale and treatment of missing values. A naïve benchmark is essential. For seasonal data, compare against seasonal naïve forecasts:

where s is the seasonal period and the formula applies when the required historical observation exists. A complex model should be adopted only if it provides meaningful, stable improvement over a suitable benchmark.

7. Worked identification example

Consider quarterly demand for a product over several years. The plot shows increasing level and a recurring fourth-quarter peak. The variance also appears larger at higher levels.

Step 1: transformation. Use z_t = ln(y_t) because demand is positive and variability is approximately proportional to level.

Step 2: differencing. The transformed series still has a trend. Apply one regular difference and one seasonal difference with s = 4:

Do not apply a second regular difference unless diagnostics show that the resulting series remains nonstationary.

Step 3: correlation patterns. Suppose the ACF of w_t has a substantial lag-1 spike and a seasonal lag-4 spike, while the PACF shows a prominent lag-1 spike and a smaller seasonal lag-4 feature. Plausible candidates include a model with non-seasonal MA(1), seasonal MA(1), or a model with corresponding AR terms. The pattern does not uniquely identify one model.

Step 4: estimation. Fit a small candidate set, for example SARIMA(0,1,1)(0,1,1)_4 and nearby models with one seasonal or non-seasonal AR term. Check convergence, parameter plausibility and information criteria.

Step 5: diagnostics. If the selected candidate has residual ACF spikes at lag 8 and a small Ljung–Box p-value, it is inadequate. Consider whether the seasonal dynamics are misspecified, whether an unusual quarter caused the pattern, or whether the transformation is unsuitable.

Step 6: validation. Use expanding-window origins and evaluate one- and four-quarter-ahead forecasts. Compare the candidate against a seasonal naïve benchmark. Select the model based on validation performance and operational requirements, not AIC alone.

Because fitting output is not supplied, no numerical parameter estimate should be invented. In practice, the analyst would report coefficients, standard errors, residual results, forecast intervals and validation metrics with the software version and data period.

8. Illustrative case: retail replenishment forecasting

Background. A regional retailer forecasts weekly sales of a household product. Sales increase during holiday periods, promotions create temporary spikes, and a stockout produces artificially low recorded sales.

Problem. The inventory team needs four-week forecasts. Under-forecasting causes stockouts; over-forecasting increases holding costs. A model trained on raw sales produces wide residual autocorrelation and misses holiday peaks.

Analysis. The analyst first confirms that weeks are equally spaced and records promotion, holiday and stockout indicators. A log transformation is considered after examining the scale–variance relationship. ACF and PACF show short-term dependence and peaks at lag 52, suggesting annual weekly seasonality. However, the available history contains only three complete annual cycles, so seasonal parameter estimates may be uncertain.

Approach. The analyst fits a small set of SARIMA models with exogenous intervention variables for promotions and stockouts, while also retaining seasonal naïve and simple exponential-smoothing benchmarks. Stockout periods are not silently treated as normal demand; they are flagged and, where possible, replaced using a documented estimate of unconstrained demand. Models are assessed through expanding-window rolling-origin evaluation at horizons 1 and 4.

Outcome. The selected model is the one with stable residuals and lower four-week MAE than the seasonal naïve benchmark. Its forecast intervals are presented to inventory managers rather than only point forecasts. The business sets a replenishment rule using the upper forecast limit and supplier lead time, while reviewing promotions that fall outside historical experience.

Learning points. Seasonality must be connected to a real operating cycle. Interventions can be more important than adding ARIMA terms. Forecast evaluation should reflect the actual decision horizon, and data quality—especially stockouts—can dominate model sophistication.

9. Limitations, governance and best practice

ARIMA-family models assume that past dependence provides useful information about future dependence. Structural breaks, new competitors, policy changes, product launches and changing customer behaviour can invalidate this assumption. Seasonal periods may also change, and missing or irregular timestamps can compromise the analysis.

Forecasts are not neutral decisions. A model that systematically under-forecasts demand for one region may allocate inventory unfairly. Governance should include documented data definitions, reproducible code, versioned models, access controls, review of sensitive variables and monitoring of forecast errors by relevant groups. Personal data should be minimised and processed under applicable privacy requirements.

Best practice includes:

• preserve a genuine out-of-sample period;
• avoid leakage from future information, including revised data unavailable at forecast time;
• use benchmarks and rolling-origin evaluation;
• inspect residuals at operationally important lags;
• report prediction intervals and their coverage over time;
• distinguish statistical significance from practical value;
• monitor parameter stability and forecast drift; and
• retain human review for unusual events without allowing undocumented overrides.

Modern systems may combine ARIMA residuals with machine-learning models, use probabilistic forecasting or automate candidate search. Automation does not remove the need for correct timestamps, leakage controls, diagnostics and accountable interpretation. Hybrid methods should demonstrate improvement in rolling-origin tests rather than being adopted because they are technologically fashionable.

Chapter Summary

ARIMA modelling begins with a defensible understanding of the series. Transformations can stabilise variance, while regular and seasonal differencing can remove stochastic trend and recurring seasonal levels. ACF and PACF provide useful but imperfect evidence for selecting AR and MA orders. ARIMA models describe non-seasonal dependence after differencing; SARIMA models add seasonal autoregressive, differencing and moving-average components.

Model adequacy requires residuals that are approximately white noise, supported by plots and tests such as Ljung–Box. Information criteria help compare in-sample fit, but rolling-origin evaluation is more relevant to forecasting performance. MAE, RMSE and carefully chosen alternatives quantify error, while seasonal naïve benchmarks provide an essential reference. Finally, data quality, structural change, interventions, governance and forecast uncertainty must be addressed alongside statistical specification.

Key Terms

Stationarity: A process whose key distributional properties do not change over time.

Transformation: A mathematical change of scale used to improve modelling properties.

Backshift operator: Operator B satisfying By_t = y_t−1.

Differencing: Subtracting observations at a specified lag to remove changing level.

Seasonal differencing: Differencing observations separated by the seasonal period.

ACF: Autocorrelation function measuring correlation at successive lags.

PACF: Partial autocorrelation after controlling for shorter lags.

AR term: Dependence of the current value on previous values.

MA term: Dependence of the current value on previous innovations.

ARIMA: Autoregressive integrated moving-average model.

SARIMA: Seasonal ARIMA model with seasonal dynamics.

Innovation: New random information entering a time series.

Invertibility: A condition supporting a unique, stable moving-average representation.

Residual: Observed value minus a model-based fitted or forecast value.

White noise: Uncorrelated random sequence with stable mean and variance.

Ljung–Box test: Joint test for residual autocorrelation over selected lags.

Rolling-origin evaluation: Repeated pseudo-out-of-sample forecasting from successive cutoffs.

MAE: Mean absolute error.

RMSE: Root mean squared error.

Seasonal naïve forecast: Forecast that repeats the observation from the corresponding prior season.

Review Questions

5 Short Answer Questions

1. What three properties define weak stationarity?
2. Why might a logarithmic transformation be applied before ARIMA modelling?
3. What is the difference between ACF and PACF?
4. What does a small Ljung–Box p-value suggest?
5. Why is a seasonal naïve model a useful benchmark?

5 Long Answer Questions

1. Explain a complete workflow for identifying, estimating and validating a SARIMA model.
2. Compare regular differencing with seasonal differencing, including their operators and risks of overuse.
3. Explain how ACF and PACF patterns support tentative ARIMA order selection.
4. Discuss why information criteria cannot replace out-of-sample forecast evaluation.
5. Explain the role of residual diagnostics in assessing model adequacy.

3 Analytical / Application Questions

1. A monthly sales series has increasing variance, a persistent trend and strong spikes at lags 12 and 24. Propose a transformation, differencing strategy and diagnostic sequence, explaining each decision.
2. A SARIMA model has lower AIC than a seasonal naïve benchmark but higher four-month rolling-origin MAE. Which model would you recommend and what further checks would you perform?
3. A retailer’s residuals show a large spike at the promotion week and apparent autocorrelation afterward. Analyse possible causes and propose modelling and governance actions.

Multiple Choice Questions with Answers

Question 1

Which condition is associated with weak stationarity?

A. A mean that changes every period

B. A variance that is constant over time

C. A seasonal period that changes randomly

D. A residual correlation that increases without limit

Correct Answer: B. A variance that is constant over time

Question 2

For quarterly data with annual seasonality, what is the usual seasonal period s?

A. 2

B. 3

C. 4

D. 12

Correct Answer: C. 4

Question 3

What does first differencing calculate?

A. y_t + y_{t−1}

B. y_t − y_{t−1}

C. y_t ÷ y_{t−1}

D. y_t × y_{t−1}

Correct Answer: B. y_t − y_{t−1}

Question 4

Which pattern is most typical of a stationary AR(p) process in the ACF and PACF?

A. ACF cuts off at p and PACF decays

B. Both ACF and PACF cut off at p

C. ACF decays and PACF cuts off after p

D. Both functions are necessarily zero at lag 1

Correct Answer: C. ACF decays and PACF cuts off after p

Question 5

In a SARIMA model, D represents:

A. The seasonal differencing order

B. The non-seasonal AR order

C. The number of forecast origins

D. The sample standard deviation

Correct Answer: A. The seasonal differencing order

Question 6

What is the main implication of a small Ljung–Box p-value?

A. Residual autocorrelation may remain

B. The model is guaranteed optimal

C. The data must be normally distributed

D. The forecast has zero error

Correct Answer: A. Residual autocorrelation may remain

Question 7

Which metric penalises large forecast errors more strongly than MAE?

A. MAPE

B. RMSE

C. Median forecast error

D. Seasonal period

Correct Answer: B. RMSE

Question 8

What is the key feature of expanding-window rolling-origin evaluation?

A. It discards all earlier observations at every origin

B. It uses future observations to fit earlier forecasts

C. It retains observations up to each successive forecast origin

D. It evaluates only in-sample fitted values

Correct Answer: C. It retains observations up to each successive forecast origin

Question 9

Why can MAPE be unsuitable when actual values are zero?

A. It requires seasonal differencing

B. It becomes undefined because of division by zero

C. It always equals RMSE

D. It cannot compare two models

Correct Answer: B. It becomes undefined because of division by zero

Question 10

What is a responsible response to a promotion-related residual spike?

A. Delete the observation without documentation

B. Ignore it because ARIMA models cannot use business information

C. Investigate it and consider a documented intervention variable

D. Automatically apply second differencing

Correct Answer: C. Investigate it and consider a documented intervention variable