Time Series • Time Series Regression Model
Time Series / Statistical Inference in Linear Regression

Statistical Inference in Linear Regression

Notes 5 Time Series Regression Model
Quick concept

After OLS produces coefficient estimates, statistical inference asks how much uncertainty remains and whether a coefficient is distinguishable from a stated null value (often zero). Inference uses the estimate, its standard error, and a test or interval. It does not replace residual checking, and it does not by itself prove that the predictor is important in practice.

Notes

Statistical Inference in Linear Regression

Definition

After OLS produces coefficient estimates, statistical inference asks how much uncertainty remains and whether a coefficient is distinguishable from a stated null value (often zero). Inference uses the estimate, its standard error, and a test or interval. It does not replace residual checking, and it does not by itself prove that the predictor is important in practice.

Example

Suppose the fitted slope of demand on temperature is positive and the associated p-value is small. That suggests, under the model assumptions, that a zero slope is a poor description of the sample. It does not by itself say that the temperature effect is large enough for operations planning.

Estimate, Uncertainty and Tests

Estimate (β̂) ↓ Uncertainty (standard error) ↓ Hypothesis test or confidence interval ↓ Interpretation (not automatic importance)
  • Coefficient estimate — the OLS number β̂ (intercept or slope).
  • Standard error (SE) — a scale for how much β̂ would typically vary from sample to sample under the model.
  • t-statistic — often (β̂ − hypothesised value) / SE. Large |t| means the estimate sits far from the null relative to its SE.
  • p-value — under the stated null and the model assumptions, how unusual the test statistic would be. A small p-value is evidence against that null, not a probability that the hypothesis is “true”.
  • Statistical significance — shorthand for “the test rejected the null at a chosen level (for example 5%)”.

A confidence interval for a coefficient is a range built from β̂ and its SE. In repeated sampling under the model, a 95% interval procedure covers the true coefficient about 95% of the time. One interval from one sample is not a 95% chance that the true value “is inside” in a mystical sense; it is the result of that procedure.

Significance vs Practical Importance

Statistical significance is not the same as practical significance. A tiny slope can be “significant” in a large sample. A large slope can fail a test in a short, noisy series. Always read the size of β̂ (and the units of x and y) together with the test.

Time-series warning: if residuals are autocorrelated, ordinary SEs, t-statistics and p-values can be misleading even when β̂ itself is a reasonable point fit. Adequacy checking comes before trusting inference.

Exam-Oriented Key Points

  1. Inference turns an estimate into a statement of uncertainty (SE, interval, test).
  2. A small p-value is evidence against a stated null, under assumptions.
  3. Do not equate statistical significance with practical importance.
  4. Autocorrelated residuals can invalidate ordinary OLS standard errors.