Time Series • Introduction to Autoregressive Models and Forecasting
Time Series / P4.06 Variable Selection

P4.06 Variable Selection

Practical 4 Introduction to Autoregressive Models and Forecasting
Quick concept

A monthly series may depend on time, a seasonal dummy for December, and a promotional flag. Compare a time-only model with a fuller model using AIC and residual plots. Do not keep useless predictors.

Practical / Solution

P4.06 Variable Selection

Problem Statement

A monthly series may depend on time, a seasonal dummy for December, and a promotional flag. Compare a time-only model with a fuller model using AIC and residual plots. Do not keep useless predictors.

Learning Outcomes

  • Fit nested regression models.
  • Use AIC as a comparison aid.
  • Prefer a simpler model when extra variables add little.

Theory

Variable selection chooses which predictors to keep. Adding variables can reduce in-sample error while fitting noise. AIC penalises extra parameters. For teaching, compare a small set of sensible candidates rather than an automated hunt through dozens of columns. Seasonal dummies and a time trend are interpretable; random extra columns are not.

Dataset / Data Source

Constructed 36 monthly observations: trend, a December lift, and a weak random promo flag that is mostly noise.

Analysis / Program

import numpy as np import pandas as pd import statsmodels.api as sm idx = pd.date_range("2023-01-01", periods=36, freq="MS") t = np.arange(36) dec = (idx.month == 12).astype(int) rng = np.random.default_rng(1) promo = rng.binomial(1, 0.2, 36) y = 40 + 0.8 * t + 12 * dec + rng.normal(0, 2, 36) def fit(cols, names): X = sm.add_constant(np.column_stack(cols)) m = sm.OLS(y, X).fit() print(names, "AIC:", round(m.aic, 1), "params:", np.round(m.params, 3)) return m m1 = fit([t], "time only") m2 = fit([t, dec], "time + December") m3 = fit([t, dec, promo], "time + December + promo")

Expected Output

Three AIC values and coefficient vectors. Time + December should improve on time only for this construction. The promo flag should look weak or unstable. Exact AIC numbers depend on the random draw.

Result / Interpretation

Keep predictors that match a real mechanism and improve AIC without wrecking residual plots. Dropping the noisy promo flag is good science, not a failure. Selection is not a substitute for checking autocorrelation.

Note

Compare a few interpretable models. Extra variables can fit noise and still leave time-series dependence in the errors.