Time Series • Introduction to Autoregressive Models and Forecasting
Time Series / Autoregressive Moving Average (ARMA) Models

Autoregressive Moving Average (ARMA) Models

Notes 4 Introduction to Autoregressive Models and Forecasting
Quick concept

An ARMA model combines an autoregressive (AR) part with a moving average (MA) part. The current observation is allowed to depend on previous values and on previous error terms (shocks). ARMA is written ARMA(p, q), where p is the AR order and q is the MA order.

Notes

Autoregressive Moving Average (ARMA) Models

Definition

An ARMA model combines an autoregressive (AR) part with a moving average (MA) part. The current observation is allowed to depend on previous values and on previous error terms (shocks). ARMA is written ARMA(p, q), where p is the AR order and q is the MA order.

Example

This month's sales may depend on last month's sales (AR) and on last month's unexpected shock, such as a one-off festival leftover (MA).

Autoregressive (AR) Part

In an AR model, the current value depends partly on previous values of the same series, plus a random error.

A simple AR(1) intuition is:

Xt = c + φ Xt−1 + εt

  • Xt = value at time t
  • Xt−1 = previous value
  • c = constant term
  • φ (phi) = AR coefficient (how strongly yesterday feeds today)
  • εt = random error (shock) at time t

If |φ| is large (but less than 1 in a weakly stationary AR(1)), the series is persistent: high values tend to stay high for a while.

Moving Average (MA) Part

In an MA model, the current value depends on the current error and on past error terms, not directly on past observed levels.

A simple MA(1) intuition is:

Xt = μ + εt + θ εt−1

  • μ (mu) = mean level
  • εt = shock at time t
  • εt−1 = previous shock
  • θ (theta) = MA coefficient

An MA(1) shock affects today and, through θ, the next period, then dies out. That is different from AR persistence through past values.

ARMA(p, q)

p is the number of lagged values in the AR part. q is the number of lagged errors in the MA part. ARMA(1, 1) uses one lagged value and one lagged error. Small p and q are preferred unless the data clearly need more.

Conceptually, ARMA is suitable when the series is stationary and shows a mix of value-dependence and shock-dependence. Stationarity here means that the typical level, the amount of variation, and the lag-dependence pattern stay reasonably stable over the sample. If the level wanders strongly, ARIMA (next topic) brings in differencing. That extra step is not part of a basic ARMA definition.

AR vs MA vs ARMA

Model Depends on Simple reading
AR Past values of the series Today follows yesterday's level, plus noise
MA Past error terms / shocks Today is affected by a recent unexpected shock
ARMA Past values and past errors Both persistence and leftover shocks matter
Exam Note

ARMA = Autoregressive + Moving Average, for a series that is already (approximately) stationary. Do not confuse ARMA with ARIMA.

Exam-Oriented Key Points

  1. AR uses lagged values; MA uses lagged errors; ARMA uses both.
  2. p is the AR order; q is the MA order.
  3. ARMA is a model for stationary behaviour, not a cure for every trend.
  4. ACF/PACF can suggest small p or q; residual checks still decide adequacy.