Pretraining vs Post-Training

9 min read

Learn

Quick Reading

Estimated 9 mins

Prereq

Intermediate

Basic ML concepts helpful

Interactive

Static Playbook

Static guide & reference tables

Two very different training regimes

Modern large language models are built in distinct stages that optimize for fundamentally different objectives. Pretraining is the stage where a model learns language, facts, and reasoning patterns from a huge, broad corpus using a simple self-supervised objective — predict the next token. Post-training is everything that happens afterward: supervised fine-tuning on curated instruction data, preference optimization (RLHF, DPO), and often additional specialized fine-tuning. The split matters because the two stages have wildly different data requirements, compute budgets, and failure modes, and conflating them leads to confused intuitions about why models behave the way they do.

Pretraining: breadth over correctness

Pretraining uses the next-token prediction objective, $\mathcal{L} = -\sum_t \log P(x_t \mid x_{<t})$, applied across trillions of tokens of raw text — web pages, books, code, forums. There's no notion of 'the right answer to a question' here; the model is simply learning the statistical structure of language and, as a byproduct, absorbing an enormous amount of world knowledge and latent reasoning capability. This stage is overwhelmingly where the compute budget goes — often 95%+ of total FLOPs for a frontier model — and it's where emergent capabilities like in-context learning and basic reasoning first appear, largely as a function of scale (data, parameters, compute) rather than any specific curation choice.

Post-training: alignment and shaping behavior

A raw pretrained model — often called a 'base model' — is a powerful but undirected text completion engine. It will happily continue a prompt in unhelpful, rambling, or unsafe ways because it was never taught to behave like an assistant; it was only taught to predict plausible continuations. Post-training fixes this with a much smaller, much more curated dataset: supervised fine-tuning (SFT) on high-quality instruction/response pairs teaches the model the format and tone of being helpful, and preference optimization (RLHF with a reward model, or direct methods like DPO) further shapes which of several plausible responses the model should prefer, using human or AI-generated preference judgments.

Pretraining vs. post-training

Dimension	Pretraining	Post-training
Objective	Next-token prediction (self-supervised)	Instruction-following, preference alignment
Data scale	Trillions of tokens, broad web/code/books	Thousands to low millions of curated examples
Data quality bar	Filtered but broad	Extremely high — every example shapes behavior directly
Compute share	~90–95%+ of total training FLOPs	Small fraction, but high-leverage
What it teaches	Language, facts, latent reasoning	Format, helpfulness, safety, tone, refusals

pipeline-stages.txt

text

Raw web/code/book corpora (trillions of tokens)
        |
        v
  [ PRETRAINING ]  -- next-token prediction, self-supervised
        |
        v
    Base model (fluent, broad, but undirected)
        |
        v
  [ SUPERVISED FINE-TUNING ]  -- curated instruction/response pairs
        |
        v
  [ PREFERENCE OPTIMIZATION ]  -- RLHF / DPO on human or AI preference data
        |
        v
    Aligned "chat" / assistant model

Post-training reveals, more than it creates, capability

A common misconception is that post-training teaches a model new skills like reasoning or coding. In most cases it's closer to the opposite: the underlying capability was already latent in the pretrained base model, and post-training's job is to make that capability reliably accessible through natural-language instructions rather than requiring careful prompt engineering. This is why base models can sometimes be coaxed, with the right few-shot prompt, into producing outputs qualitatively similar to the aligned model's — the knowledge was there, just not reliably surfaced by default.

Post-training can also erase capability

Aggressive post-training, especially heavy-handed safety tuning or narrow preference optimization, can measurably degrade capabilities the base model had — a phenomenon sometimes called the 'alignment tax.' Over-fitting to a narrow style of 'ideal' responses during SFT can make a model worse at tasks that require a different response style than the fine-tuning data emphasized, which is why post-training datasets are curated with as much care as pretraining corpora, just at a very different scale.

What's next?

Understanding this split clarifies where most of the interesting day-to-day engineering work on LLM behavior actually happens — not in redoing a trillion-token pretraining run, but in the much more tractable, iterative loop of curating post-training data and preference signals. The next natural question is what happens when you need to teach a model genuinely new knowledge after it's already been through both stages.

I build these systems professionally.

Whether it's a RAG pipeline, analytics migration, or AI workflow — let's talk.

Work With Me →