The major deep generative model families each sacrifice something. GANs produce sharp samples but have no way to compute how likely a given data point is, and training is notoriously unstable. VAEs give you a likelihood — but only a lower bound (the ELBO), because the encoder's approximate posterior isn't exact. Normalizing Flows are the one family that gives you exact, tractable likelihoods, by construction, at the cost of architectural restrictions on what the network is allowed to compute.

The core trick: invertible transformations and change of variables

Start with a simple distribution you understand completely — usually a standard Gaussian. A Normalizing Flow is a sequence of invertible, differentiable transformations f = f_k ∘ ... ∘ f_1 that maps samples z from that simple distribution into samples x = f(z) from a complex distribution that matches your data.

Because every f_i is invertible, you can go the other direction too: given a real data point x, you can compute exactly which z produced it (z = f⁻¹(x)), and — crucially — the change-of-variables formula tells you exactly how the probability density transforms under that mapping. That means you get an exact log-likelihood for any real data point, not an approximation.

python

Note

For an arbitrary neural network layer, computing the determinant of its Jacobian costs O(n³) for an n-dimensional input — intractable for real data. Every practical flow architecture (RealNVP, NICE, Glow) is specifically designed so the Jacobian is triangular, making its determinant trivial to compute (just the product of diagonal entries). This single constraint — cheap, exact Jacobians — is the reason flow architectures look the way they do: most of a flow layer's design is really just satisfying this requirement.

Coupling layers: the standard building block

RealNVP's coupling layer splits the input into two halves. One half passes through unchanged. The other half is transformed by a scale-and-shift function whose parameters are computed from the unchanged half. This construction is trivially invertible (just reverse the scale-and-shift algebraically) and its Jacobian is triangular by design, so its determinant is cheap. Stacking many coupling layers, alternating which half is held fixed, lets the overall flow become an arbitrarily expressive transformation while keeping every individual layer easy to invert and differentiate.

Normalizing Flows vs. VAEs vs. GANs

Property	Normalizing Flow	VAE	GAN
Exact likelihood	Yes	No — only a lower bound (ELBO)	No likelihood at all
Latent dimension	Must equal data dimension	Can be smaller (a bottleneck)	Arbitrary, chosen freely
Sample quality	Good, often less sharp than GANs	Often blurry	Typically sharpest
Training stability	Stable (direct likelihood maximization)	Stable	Often unstable
Invertible / exact reconstruction	Yes, exactly	No — lossy reconstruction	No

The dimensionality constraint

Because every flow layer must be invertible, the latent space z must have exactly the same dimensionality as the data x — you cannot compress to a smaller bottleneck the way a VAE does. This is a real architectural limitation: flows can't do the kind of dimensionality reduction that makes VAEs useful for compact representations, and it means flow models tend to be larger and more compute-intensive per sample than comparable VAEs.

Note

Normalizing flows are the backbone of several production audio systems — WaveGlow and Glow-based vocoders generate speech waveforms with exact likelihood training. They're used in scientific simulation (sampling from complex physical distributions, lattice QCD) and density estimation for anomaly detection, where having an actual probability — not just a sample — matters. They've been less dominant in image and text generation, where diffusion models and autoregressive transformers now lead, partly because flows' dimensionality constraint makes them less efficient at very high resolutions.

What's next

Flows get exact likelihoods by restricting the architecture to invertible transformations. Energy-Based Models take the opposite approach: give up on a normalized probability entirely, and instead learn a flexible energy function that can be literally any neural network, with no invertibility constraint at all.

I build these systems professionally.

Whether it's a RAG pipeline, analytics migration, or AI workflow — let's talk.

Work With Me →

Need production AI or MarTech expertise? I build custom LLM workflows, server-side GTM tagging, and analytics infrastructures professionally.

Need custom AI or MarTech setup? Let's build together.

Let's talk →