ARIMA Models Explained

9 min read

Learn

Quick Reading

Estimated 9 mins

Prereq

Intermediate

Basic ML concepts helpful

Interactive

Static Playbook

Static guide & reference tables

ARIMA: three ideas combined

ARIMA stands for AutoRegressive Integrated Moving Average, and the name literally describes its three components, controlled by the parameters $(p, d, q)$:

AR($p$) — Autoregressive: the current value is a linear combination of the previous $p$ values.

I($d$) — Integrated: the series is differenced $d$ times to make it stationary before modeling (see the stationarity module for why this matters).

MA($q$) — Moving Average: the current value also depends on a linear combination of the previous $q$ forecast errors (not raw values — the model's own past mistakes).

Put together, ARIMA($p,d,q$) models the $d$-th difference of the series as a function of its own past values and past error terms.

The AR and MA components in equations

For a stationary series $y_t$ (after differencing $d$ times), the AR($p$) component is:

$y_t = c + \phi_1 y_{t-1} + \phi_2 y_{t-2} + \dots + \phi_p y_{t-p} + \epsilon_t$

where $\phi_i$ are the autoregressive coefficients and $\epsilon_t$ is white noise.

The MA($q$) component adds dependence on past error terms:

$y_t = c + \epsilon_t + \theta_1 \epsilon_{t-1} + \theta_2 \epsilon_{t-2} + \dots + \theta_q \epsilon_{t-q}$

A full ARIMA($p,d,q$) model combines both on the differenced series $y_t' = \Delta^d y_t$:

$y_t' = c + \sum_{i=1}^{p} \phi_i y_{t-i}' + \epsilon_t + \sum_{j=1}^{q} \theta_j \epsilon_{t-j}$

Setting $q=0$ gives a pure AR model; setting $p=0$ gives a pure MA model; $d=0$ means no differencing was needed (the series was already stationary).

Choosing (p, d, q): ACF, PACF, and information criteria

$d$ is chosen by testing stationarity (e.g. the ADF test) and differencing until the series passes.

$p$ and $q$ are traditionally chosen by inspecting the autocorrelation function (ACF) and partial autocorrelation function (PACF) of the differenced series. A sharp cutoff in the PACF after lag $p$ with a slowly decaying ACF suggests a pure AR($p$) process; the mirror pattern (sharp ACF cutoff, decaying PACF) suggests a pure MA($q$) process. In practice, most real series need both components, and analysts increasingly let automated search pick the orders by minimizing an information criterion like AIC or BIC, which penalize both poor fit and excessive complexity.

Fitting ARIMA with statsmodels

python

import pandas as pd
from statsmodels.tsa.arima.model import ARIMA
from statsmodels.tsa.stattools import adfuller

# series: pandas Series indexed by date, already checked/made stationary
# if adfuller(series)[1] > 0.05, difference before or let d handle it

model = ARIMA(series, order=(2, 1, 1))  # p=2, d=1, q=1
fit = model.fit()

print(fit.summary())

forecast = fit.get_forecast(steps=12)
mean_forecast = forecast.predicted_mean
conf_int = forecast.conf_int(alpha=0.05)

print("Next 12 forecasts:")
print(mean_forecast)
print("95% confidence interval:")
print(conf_int)

Automatic order selection with pmdarima

python

import pmdarima as pm

# auto_arima searches over (p, d, q) combinations, selecting by AIC,
# and also tests differencing order d automatically via ADF/KPSS
auto_model = pm.auto_arima(
    series,
    start_p=0, start_q=0,
    max_p=5, max_q=5,
    d=None,              # let the function determine d
    seasonal=False,
    information_criterion="aic",
    stepwise=True,
    suppress_warnings=True,
)

print(auto_model.summary())
forecast, conf_int = auto_model.predict(n_periods=12, return_conf_int=True)

SARIMA extends ARIMA to seasonal data

Plain ARIMA assumes no repeating seasonal pattern. SARIMA($p,d,q)(P,D,Q)_m$ adds a second set of seasonal AR, differencing, and MA terms operating at lag $m$ (the season length), on top of the regular ones. For monthly retail data with yearly seasonality, $m=12$. statsmodels.tsa.statespace.sarimax.SARIMAX and pmdarima's seasonal=True option both support this directly, and it's the right default whenever a series shows a visible repeating cycle in its ACF plot.

Residuals should look like white noise

After fitting, always check the residuals: plot them, check their ACF, and run a formal test like the Ljung-Box test. If residuals still show significant autocorrelation, the model hasn't captured all the structure in the series — a sign $p$ or $q$ needs adjusting, or that the series has structure (seasonality, structural breaks) ARIMA alone can't represent.

What's Next?

ARIMA models the data by regressing on its own past values and errors, which gives it flexibility but little interpretability about trend and seasonal components individually. An alternative family of methods builds forecasts by explicitly smoothing and extrapolating level, trend, and seasonal components separately — often simpler to reason about and just as effective in practice.

I build these systems professionally.

Whether it's a RAG pipeline, analytics migration, or AI workflow — let's talk.

Work With Me →