Time Series • Introduction to Autoregressive Models and Forecasting
Time Series / P4.07 Weighted Least Squares

P4.07 Weighted Least Squares

Practical 4 Introduction to Autoregressive Models and Forecasting
Quick concept

When later months are noisier than early months, ordinary OLS treats every point equally. Fit WLS with weights that down-weight high-variance periods and compare the slope to OLS.

Practical / Solution

P4.07 Weighted Least Squares

Problem Statement

When later months are noisier than early months, ordinary OLS treats every point equally. Fit WLS with weights that down-weight high-variance periods and compare the slope to OLS.

Learning Outcomes

  • Explain why unequal error variance is a problem for OLS.
  • Fit WLS with known or estimated weights.
  • Compare OLS and WLS slopes.

Theory

Weighted least squares minimises Σ w_t (y_t − x_t′β)². If Var(ε_t) is larger for some t, those points should get smaller weight (often w_t = 1/σ_t²). WLS addresses heteroscedasticity, not autocorrelation. Do not treat WLS as a cure for lagged errors.

Dataset / Data Source

Constructed 30-point series whose noise scale grows with time. Teaching data.

Analysis / Program

import numpy as np import statsmodels.api as sm rng = np.random.default_rng(0) t = np.arange(30) sigma = 0.5 + 0.15 * t y = 10 + 0.4 * t + rng.normal(0, 1, 30) * sigma X = sm.add_constant(t) ols = sm.OLS(y, X).fit() wls = sm.WLS(y, X, weights=1 / sigma**2).fit() print("OLS slope:", round(ols.params[1], 3), "SE:", round(ols.bse[1], 3)) print("WLS slope:", round(wls.params[1], 3), "SE:", round(wls.bse[1], 3))

Expected Output

Printed OLS and WLS slopes with their standard errors. They will be similar but not identical. WLS SEs use the weights; they are still not valid if residuals are serially correlated.

Result / Interpretation

WLS gives more influence to precise early points. That is appropriate when variance clearly grows. If the real problem is autocorrelation, WLS is the wrong tool and GLS or a lagged-error model is next.

Note

WLS handles unequal variances. It does not remove serial correlation.