Time-Series Regression and Dynamic Models

Introduction

Time-series regression combines the explanatory logic of regression with the temporal structure of observed data. It is useful when an analyst wants not only to extrapolate past patterns but also to explain how demand, sales, energy use, costs, or other outcomes respond to calendar effects, interventions, prices, promotions, weather, economic conditions, and delayed influences.

A time-series regression model differs from ordinary cross-sectional regression in an important respect: observations ordered in time are often dependent. Errors may be correlated, predictors may trend together, and future predictor values may be unknown. Ignoring these features can produce misleading standard errors, unstable coefficients, and poor forecasts.

This chapter develops practical models ranging from trend and seasonal-indicator regression to dynamic regression and ARIMAX. The central principle is that a useful forecasting model must represent both the systematic component of the series and the dependence remaining after that component has been explained.

Learning objectives

By the end of this chapter, you should be able to:

1. Specify regression models for trend and seasonality.
2. Use seasonal indicators, Fourier terms, and intervention variables appropriately.
3. Explain distributed lags and external predictors.
4. Diagnose and model autocorrelated regression errors.
5. Distinguish static regression, dynamic regression, and ARIMAX.
6. Recognize multicollinearity and predictor uncertainty.
7. Evaluate models using time-ordered validation and forecast-oriented measures.
8. Communicate model limitations and governance implications.

1. The regression view of a time series

Let y_t denote the observed outcome at time t. A basic regression representation is

Here, β₀ is the intercept, β_j is the partial effect of predictor x_j,t, and ε_t is the error at time t. The equation is meaningful when the predictors and outcome are measured consistently and the relationship is sufficiently stable over the modeling period.

In ordinary regression, errors are commonly assumed to have conditional mean zero and constant variance. For time series, the stronger assumption of independence is often unrealistic. A regression coefficient can still be useful when errors are autocorrelated, but conventional standard errors are then generally unreliable and forecasts may omit predictable residual structure.

A useful decomposition is:

where m_t is the systematic component explained by predictors and n_t is a stochastic component. Regression models estimate m_t; dynamic error models describe dependence in n_t.

Trend terms

A linear trend model is

where t is a time index. The coefficient β₁ is the expected change in the outcome per time unit, holding other variables constant. A quadratic trend extends this to

The quadratic term should be used only when curvature is substantively plausible and supported by validation. High-order polynomials can behave implausibly at the forecast horizon, particularly near or beyond the range of the training data. Centering time, for example by using t_c = t − \bar{t}, can reduce correlation between the intercept and trend terms; it does not eliminate all multicollinearity.

Trend may also be represented by splines or segmented terms when growth changes at known or estimated points. Such flexibility must be controlled to avoid fitting historical noise.

2. Representing seasonality

Seasonality is a recurring pattern associated with calendar position, such as month, weekday, quarter, or hour. For monthly data with a stable seasonal pattern, one month is selected as a reference category and the other 11 months are represented by indicators:

D_m,t equals 1 when observation t belongs to month m, and 0 otherwise. The reference month is represented by all indicators equal to zero. γ_m measures the expected difference from the reference month, conditional on trend and other predictors.

Including all 12 month indicators together with an intercept creates perfect multicollinearity because their sum equals one. This is the dummy-variable trap. Alternatives are to omit the intercept or impose a sum-to-zero constraint, but the interpretation changes accordingly.

Seasonal indicators are flexible and interpretable, but they require many parameters for high-frequency data. They also assume that the seasonal effect is broadly stable unless interactions or changing coefficients are introduced.

Fourier terms

Smooth seasonality can be represented with sine and cosine pairs:

Here, m is the seasonal period in observations, K is the number of harmonics, and a_k and b_k are estimated coefficients. For monthly annual seasonality, m = 12. The first harmonic captures one broad annual cycle; additional harmonics capture more detailed shapes. Typically, K ≤ floor(m/2), and a smaller value is often preferable for forecasting.

Fourier terms are especially useful for long seasonal periods, such as daily data with weekly or annual cycles. They provide a parsimonious smooth representation, but their coefficients are less directly interpretable than month indicators. The seasonal period must be specified correctly, and multiple seasonalities require separate Fourier sets.

3. Interventions and structural change

An intervention is an event that may alter the level or trajectory of a series. Examples include a product launch, policy change, supply disruption, store closure, or advertising campaign.

A pulse variable represents a temporary effect:

where P_t = 1 only during the event period. A step variable represents a permanent level shift after a date T₀:

A slope-change intervention can be defined as

and included as θR_t. Before the intervention, R_t = 0; afterward, it increases by one each period. A model containing both S_t and R_t permits an immediate level change and a changed trend.

Intervention coefficients should not automatically be interpreted as causal effects. A policy may coincide with macroeconomic change, competitor action, or measurement changes. Causal interpretation requires a credible design, such as a control series, interrupted time-series assumptions, or difference-in-differences structure.

4. External predictors and distributed lags

An external predictor, or exogenous variable, is a variable used to explain the outcome but not generated by the outcome model being estimated. Examples include temperature, price, exchange rate, web traffic, and planned marketing expenditure. A contemporaneous model is

The coefficient β₁ describes the conditional association between x_t and y_t at the same time point. Its interpretation requires care when both variables respond to common shocks or when reverse causality is possible.

Effects often occur over several periods. A finite distributed-lag model is

where q is the maximum lag. β₀ is the contemporaneous effect, while β_j is the effect associated with predictor values j periods earlier, conditional on the other included lags. The cumulative effect of a one-unit increase occurring across the modeled lag window is

provided the coefficients are on a common scale and the intervention or change is interpreted consistently.

For example, suppose weekly demand is modeled as y_t = 100 + 4x_t + 3x_t−1 + ε_t, where x is advertising expenditure measured in thousands of dollars. A one-unit increase in current advertising is associated with 4 additional units of demand currently and 3 additional units next week, conditional on the rest of the model. The modeled cumulative association is 7 demand units. This is not necessarily a causal return on advertising.

Long lag structures may be summarized with polynomial or geometric restrictions, but restrictions should be justified and validated. Lagged predictors can be highly correlated, especially when x_t changes slowly. This creates unstable individual coefficients even when their combined predictive effect is reasonably estimated.

5. Regression with autocorrelated errors

Suppose the regression mean is correctly specified but the error follows an autoregressive process:

where η_t is an innovation with mean zero and constant variance, and the autoregressive process is stationary when its roots satisfy the relevant stationarity condition. In the AR(1) case, a sufficient condition is |φ₁| < 1.

The complete model can be written as

This is regression with autocorrelated errors. The predictors explain the conditional mean, while the error model explains serial dependence not captured by predictors.

Ignoring autocorrelation often leaves residual patterns and can make confidence intervals too narrow. Ordinary least squares coefficients may remain unbiased under strict exogeneity, but efficiency and inference are affected; with lagged outcomes, feedback, omitted variables, or endogenous predictors, bias can also occur.

A practical workflow is:

1. Fit a substantively specified regression.
2. Inspect residual time plots and autocorrelation and partial-autocorrelation functions.
3. Test candidate error structures using diagnostics and information criteria, without relying mechanically on one test.
4. Refit using generalized least squares, maximum likelihood, or a regression-with-ARIMA-errors procedure.
5. Check whether standardized residuals resemble uncorrelated noise.

Residual autocorrelation can also indicate missing seasonality, omitted lags, structural breaks, or an incorrect functional form. Adding an AR error term should not be used to conceal a poorly specified mean model.

6. Dynamic regression and ARIMAX

Dynamic regression extends regression by incorporating lagged outcomes and/or lagged predictors. A simple autoregressive distributed-lag model is

The lagged outcome represents persistence. Unlike a model with only an exogenous predictor, the effect of a predictor may propagate through future values of y_t. For stability, the dynamic component must satisfy appropriate stationarity or boundedness conditions, unless the model is deliberately specified for nonstationary data with valid cointegration reasoning.

ARIMAX commonly refers to a regression model with ARIMA errors:

where the error process n_t follows an ARIMA model, often written using the backshift operator B as

Here, B n_t = n_t−1, d is the differencing order, φ(B) is the autoregressive polynomial, θ(B) is the moving-average polynomial, and e_t is white noise. Seasonal ARIMA terms may also be added.

The distinction between ARIMAX with ARIMA errors and an autoregressive distributed-lag model matters. In the former, autocorrelation is explicitly assigned to the error process after accounting for predictors. In the latter, lagged outcomes are included directly in the conditional mean. Software packages sometimes use the term ARIMAX inconsistently, so the fitted equation and forecasting procedure should be documented.

Forecasting requires future values of every predictor in x_t. If future temperature, price, or advertising is unknown, the analyst must supply forecasts, planned values, scenarios, or a joint model. Treating unknown future predictors as known can make forecast intervals unrealistically narrow.

7. Multicollinearity and predictor uncertainty

Multicollinearity occurs when predictors contain overlapping information. Common causes include a time trend with seasonal indicators, current and lagged advertising, multiple economic indicators, and Fourier terms combined with flexible trends.

Consequences include inflated coefficient variances, unstable signs and magnitudes, sensitivity to small data changes, and difficulty attributing effects to individual predictors. Multicollinearity does not necessarily reduce out-of-sample forecast accuracy, but it weakens interpretation and can make scenario analysis unreliable.

Useful responses include:

• Remove redundant predictors using substantive reasoning.
• Combine related variables into an index.
• Use fewer lags or a justified lag constraint.
• Center or scale variables when appropriate.
• Use regularization such as ridge regression for prediction.
• Use principal components when interpretability is not the main objective.
• Collect more informative data or extend the time span.

A variance-inflation statistic is commonly defined for predictor j as

where R_j² is obtained by regressing predictor x_j on the other predictors. It measures linear redundancy, not forecast quality, and threshold rules such as 5 or 10 are heuristics rather than universal laws.

Predictor uncertainty has two distinct sources. First, future predictor values may be unknown. Second, estimated regression and time-series parameters are uncertain. A conditional forecast treats future predictors as fixed; a predictive forecast should incorporate uncertainty in both outcomes and predictors. Scenario forecasts are often more honest for managerial decisions: for example, produce demand forecasts under low, expected, and high advertising plans.

8. Forecast evaluation and implementation

Random train-test splitting is inappropriate for most time series because it allows information from the future to influence the past. Use a chronological holdout or rolling-origin evaluation. In rolling-origin evaluation, the model is repeatedly trained on an expanding or moving window and forecasts the next one or more periods.

Common measures include mean absolute error:

and root mean squared error:

where h is the forecast horizon, y is the observed value, and ŷ is the forecast made using information available at the forecast origin. MAE is in the outcome’s units; RMSE penalizes large errors more heavily. Percentage measures can be problematic when actual values are zero or close to zero.

Evaluation should compare the proposed model with sensible benchmarks, such as seasonal naïve forecasts. A complex dynamic regression should earn its complexity through better forecasts, better decisions, or clearer explanation. Residual checks remain essential: residuals should have approximately zero mean, stable variance where assumed, and little remaining autocorrelation. Prediction intervals should be calibrated, not merely narrow.

Illustrative case: forecasting weekly grocery demand

Background and problem

A grocery chain wants to forecast weekly demand for a chilled product across 104 weeks. Demand is affected by a long-run trend, holiday seasonality, promotions, price, and temperature. Management needs forecasts four weeks ahead to plan inventory.

Analysis

The analyst first plots demand and finds annual seasonality and a possible level change after a packaging redesign in week 60. Promotion expenditure has effects in the current and following week. Price and promotion are negatively associated because discounts are often paired with advertising, creating multicollinearity.

Approach and solution

The candidate mean model is

where seasonal_t is represented by Fourier terms, S_t is a step intervention beginning at week 60, and ε_t is modeled with an AR(1) error. The analyst standardizes predictors for numerical stability, examines correlations and VIF values, and compares the model with a seasonal-naïve benchmark and a model using monthly indicators.

For four-week forecasting, future prices and promotion plans are supplied by the commercial team. Three scenarios are generated: planned promotion, 10% lower promotion, and 10% higher promotion. Rolling-origin validation shows that the dynamic regression has lower MAE than the benchmark, while the indicator model has similar accuracy but wider coefficient uncertainty. The final report emphasizes scenario-dependent forecasts rather than one supposedly certain number.

Outcome and learning points

The model supports procurement planning but does not prove that promotion caused all additional demand. A major limitation is that promotion decisions respond to expected demand, creating possible endogeneity. The analyst recommends a future randomized promotion test or a control-store design. The case illustrates four principles: model seasonality parsimoniously, represent delayed effects explicitly, validate in time order, and disclose uncertainty in both predictors and parameters.

Professional, ethical, and technological considerations

Forecasts influence inventory, staffing, pricing, credit, and public services. Analysts should document data definitions, transformations, missing-value treatment, forecast origins, model versions, and assumptions about future predictors. Structural breaks should be investigated rather than silently overwritten.

Fairness concerns arise when external predictors encode unequal treatment or when forecasts allocate scarce resources across communities. A model may be statistically accurate overall yet systematically inaccurate for smaller segments. Monitoring should therefore include subgroup errors where legally and ethically appropriate.

Modern forecasting platforms can automate lag selection, hyperparameter search, and probabilistic forecasting. Automation does not remove the need for leakage controls, temporal validation, interpretability, and human review. Machine-learning models may improve accuracy, while regression and dynamic models often provide stronger transparency and scenario capability. Hybrid systems can combine machine-learning predictors with explicit residual time-series models, but the full pipeline must be validated as one forecasting system.

Best-practice checklist

• Define the forecast horizon and information set before fitting the model.
• Plot the outcome, predictors, missingness, and unusual events.
• Encode trend and seasonality according to the data-generating context.
• Include lags only when operationally and substantively justified.
• Check residual autocorrelation after fitting the mean model.
• Avoid interpreting unstable coefficients as precise causal effects.
• Forecast or scenario-plan every predictor required in the future.
• Use rolling-origin validation and compare with simple benchmarks.
• Report intervals and assumptions, not only point forecasts.
• Monitor performance after deployment and investigate structural change.

Chapter Summary

Time-series regression explains an outcome using trend, seasonal structure, interventions, and external predictors while recognizing that observations over time are dependent. Seasonal indicators offer flexible categorical effects; Fourier terms provide compact smooth seasonality. Interventions represent temporary or permanent changes, and distributed lags capture delayed predictor effects. Autocorrelated-error models and ARIMAX separate the systematic regression component from remaining serial dependence. Dynamic regression may additionally include lagged outcomes.

Multicollinearity can destabilize individual coefficients, especially when trends, lagged predictors, and seasonal representations overlap. Future predictor uncertainty must be addressed through forecasts, scenarios, or joint modeling. Reliable implementation requires chronological validation, residual diagnostics, benchmark comparisons, calibrated intervals, and transparent governance. The best model is not necessarily the most complex; it is the model whose assumptions, forecasts, and limitations are fit for the decision.

Key Terms

Time-series regression: Regression applied to observations ordered in time.

Trend: Systematic long-run movement in a series.

Seasonality: A recurring pattern linked to calendar position.

Seasonal indicator: A binary variable representing a calendar category.

Fourier term: A sine or cosine component used to model smooth cycles.

Intervention: An event or change represented explicitly in a model.

Pulse variable: An indicator for a temporary event effect.

Step variable: An indicator for a persistent level change.

Distributed lag: A model distributing a predictor’s effect across time lags.

External predictor: A variable used to explain or forecast an outcome.

Autocorrelation: Dependence between observations or errors at different times.

Autoregressive error: An error process related to its own previous values.

Dynamic regression: Regression containing lagged outcomes and/or predictors.

ARIMAX: Regression with external predictors and ARIMA-type errors.

Multicollinearity: Strong linear overlap among predictors.

VIF: A diagnostic measuring predictor redundancy through an auxiliary regression.

Rolling-origin evaluation: Repeated time-ordered forecast testing.

Forecast horizon: Number of future periods being forecast.

Predictor uncertainty: Uncertainty about future predictor values or their estimates.

Structural break: A change in the data-generating relationship over time.

Review Questions

Short Answer Questions

1. Why can ordinary regression assumptions be problematic for time-series data?
2. What is the difference between a pulse intervention and a step intervention?
3. Why are Fourier terms useful for long seasonal periods?
4. What does a distributed-lag model represent?
5. Why must future external predictors be specified when producing forecasts?

Long Answer Questions

1. Explain how seasonal indicators and Fourier terms differ in representation, interpretation, and parameter efficiency.
2. Describe a complete workflow for diagnosing and modeling autocorrelated regression errors.
3. Compare static regression, dynamic regression, and ARIMAX.
4. Explain the causes and consequences of multicollinearity in time-series regression and evaluate possible remedies.
5. Discuss why rolling-origin evaluation and forecast intervals are important in professional forecasting.

Analytical / Application Questions

1. A retailer observes weekly demand, price, promotion, and temperature. Design a regression or dynamic-regression specification, including plausible lags and seasonal terms, and justify each component.
2. A policy begins in month 25 and may cause both an immediate level change and a gradual trend change. Specify suitable intervention variables and explain how their coefficients would be interpreted.
3. A model has excellent historical fit but poor rolling-origin forecasts because future advertising is unknown. Propose a scenario-based forecasting procedure and identify the main governance risks.

Multiple Choice Questions with Answers

Question 1

Which representation is most appropriate for a temporary one-period event?

A. A pulse variable

B. A permanent step variable

C. A quadratic trend

D. A seasonal reference category

Correct Answer: A. A pulse variable

Question 2

In a monthly seasonal-indicator model with an intercept, why is one month usually omitted?

A. To remove autocorrelation

B. To avoid perfect multicollinearity

C. To guarantee normal residuals

D. To make the series stationary

Correct Answer: B. To avoid perfect multicollinearity

Question 3

For a distributed-lag model with coefficients 4 and 3 on current and one-period-lagged advertising, what is the modeled cumulative effect?

A. 1

B. 3

C. 4

D. 7

Correct Answer: D. 7

Question 4

What does an AR(1) error model describe?

A. The outcome’s seasonal indicators

B. The relationship between an error and its previous error

C. The correlation between two predictors only

D. The cumulative effect of a distributed lag

Correct Answer: B. The relationship between an error and its previous error

Question 5

Which validation design best respects temporal ordering?

A. Randomly assigning all observations to folds

B. Using only the first observation as the test set

C. Rolling-origin evaluation

D. Removing the most recent observations

Correct Answer: C. Rolling-origin evaluation

Question 6

If R_j² = 0.80, what is VIF_j?

A. 0.20

B. 1.25

C. 4

D. 5

Correct Answer: D. 5

Question 7

What is a principal forecasting implication of an unknown future predictor?

A. The predictor can be ignored without consequence

B. A forecast, scenario, or joint model for that predictor is needed

C. The residual variance becomes zero

D. Seasonality must be removed

Correct Answer: B. A forecast, scenario, or joint model for that predictor is needed

Question 8

Which statement best describes Fourier terms?

A. They model seasonality with sine and cosine components

B. They always provide causal interpretations

C. They eliminate the need for residual diagnostics

D. They are identical to intervention variables

Correct Answer: A. They model seasonality with sine and cosine components

Question 9

What is a common consequence of multicollinearity?

A. More stable individual coefficients

B. Inflated coefficient uncertainty

C. Guaranteed lower forecast error

D. Automatic stationarity

Correct Answer: B. Inflated coefficient uncertainty

Question 10

In an ARIMAX model, what does the ARIMA component commonly model?

A. The remaining error process after predictors are included

B. Only the calendar labels

C. The predictor measurement units

D. The reference category for seasonality

Correct Answer: A. The remaining error process after predictors are included