Time Series • Introduction to Autoregressive Models and Forecasting

P4.02 Least Squares Estimation

Practical 4 Introduction to Autoregressive Models and Forecasting

Quick concept

Compute OLS intercept and slope for a tiny six-point series both with the normal equations and with statsmodels, then confirm they match.

Practical / Solution

P4.02 Least Squares Estimation

Problem Statement

Compute OLS intercept and slope for a tiny six-point series both with the normal equations and with statsmodels, then confirm they match.

Learning Outcomes

State the least-squares criterion.
Estimate β0 and β1 from formulas.
Check the software estimate against the formulas.

Theory

Least squares chooses coefficients that minimise Σ(y_t − β0 − β1 x_t)². For simple regression, β1 = Σ(x − x̄)(y − ȳ) / Σ(x − x̄)² and β0 = ȳ − β1 x̄. The fitted line passes through the point of means. This is a calculation practical: students should see that the software is doing the same arithmetic.

Dataset / Data Source

Six observations: t = 1..6 and y = 10, 12, 13, 15, 16, 18. Teaching data small enough to compute by hand.

Analysis / Program

import numpy as np import statsmodels.api as sm x = np.arange(1, 7, dtype=float) y = np.array([10, 12, 13, 15, 16, 18], dtype=float) # Least-squares formulas xbar, ybar = x.mean(), y.mean() beta1 = np.sum((x - xbar) * (y - ybar)) / np.sum((x - xbar) ** 2) beta0 = ybar - beta1 * xbar print("Hand slope:", round(beta1, 4)) print("Hand intercept:", round(beta0, 4)) # Same fit with OLS X = sm.add_constant(x) fit = sm.OLS(y, X).fit() print("Software intercept, slope:", np.round(fit.params, 4))

Expected Output

Printed intercept and slope from the formulas and the same pair from statsmodels. For this six-point table the two methods must agree to rounding error. Students can also compute the slope on paper.

Result / Interpretation

Agreement shows that 'least squares' is a defined calculation, not a mysterious black box. Later practicals add standard errors and residual plots on top of these same coefficients.

Note

OLS estimates minimise squared residuals. Matching the formula to software builds trust in the numbers.