Time Series • Time Series Regression Model
Time Series / P5.05 Compare Multi-Variable Forecasts

P5.05 Compare Multi-Variable Forecasts

Practical 5 Time Series Regression Model
Quick concept

Compare a multivariate VAR(1) forecast with separate last-value naive forecasts for each of two series. Report MAE for each method and each variable.

Practical / Solution

P5.05 Compare Forecasts Across Variables

Problem Statement

Compare a multivariate VAR(1) forecast with separate last-value naive forecasts for each of two series. Report MAE for each method and each variable.

Learning Outcomes

  • Build a fair time-based hold-out.
  • Compare multivariate vs naive univariate forecasts.
  • Read which variable was easier to forecast.

Theory

A multivariate model is useful only if it improves decisions or accuracy relative to a simple alternative. Last-value naive forecasts are a honest baseline for persistent series. Compare MAE (and RMSE if scale comparison helps) on the same hold-out. A win on one variable and a loss on the other is a real outcome, not a failure of the exercise.

Dataset / Data Source

Reuse the two-series construction from P5.04 (same seed so students can compare).

Kaggle-style Workflow

  • Split by time
  • Naive forecast per column
  • VAR forecast
  • MAE table
  • Interpret which method wins on this sample

Analysis / Program

import numpy as np import pandas as pd from statsmodels.tsa.api import VAR rng = np.random.default_rng(4) n = 90 e1, e2 = rng.normal(size=n), rng.normal(size=n) x = np.zeros(n) y = np.zeros(n) for t in range(1, n): x[t] = 0.5 * x[t-1] + 0.2 * y[t-1] + e1[t] y[t] = 0.4 * y[t-1] + 0.1 * x[t-1] + e2[t] df = pd.DataFrame({"x": x, "y": y}) train, test = df.iloc[:-6], df.iloc[-6:] naive = pd.DataFrame({ "x": np.repeat(train["x"].iloc[-1], 6), "y": np.repeat(train["y"].iloc[-1], 6) }, index=test.index) fc = pd.DataFrame( VAR(train).fit(1).forecast(train.values[-1:], 6), index=test.index, columns=["x", "y"] ) def mae(a, b): return np.mean(np.abs(a - b)) print("Naive MAE x, y:", round(mae(test.x, naive.x), 3), round(mae(test.y, naive.y), 3)) print("VAR MAE x, y:", round(mae(test.x, fc.x), 3), round(mae(test.y, fc.y), 3))

Expected Output

Four MAE numbers: naive and VAR for x and for y. Which method wins can change with the random seed; the point is the comparison table, not a guaranteed VAR victory.

Result / Interpretation

Comparing methods prevents treating a multivariate fit as automatically better. If VAR MAE is close to naive, the cross-variable lags may be weak in this sample. Report both variables; do not hide the worse one.

Note

Always compare multivariate forecasts with a simple baseline on unused dates.