> ## Documentation Index
> Fetch the complete documentation index at: https://aegean.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Normalizing flows and exact likelihoods

> Exact densities of generated samples from invertible maps.

[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/pantelis/eng-ai-agents/blob/main/notebooks/normalizing-flows/index.ipynb)

```python theme={null}
import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import norm

# Source p_Z is the standard normal N(0, 1). Target p_X is this Gaussian mixture.
PI = np.array([0.3, 0.5, 0.2])
MU = np.array([-2.0, 1.0, 3.5])
SIGMA = np.array([0.4, 0.6, 0.3])
U_CLIP = 1e-12  # quantile levels are clipped to [U_CLIP, 1 - U_CLIP]


def mixture_pdf(x, pi=PI, mu=MU, sigma=SIGMA):
    x = np.asarray(x, dtype=float)[..., None]
    return np.sum(pi * norm.pdf(x, mu, sigma), axis=-1)


def mixture_cdf(x, pi=PI, mu=MU, sigma=SIGMA):
    x = np.asarray(x, dtype=float)[..., None]
    return np.sum(pi * norm.cdf(x, mu, sigma), axis=-1)


def mixture_ppf(u, pi=PI, mu=MU, sigma=SIGMA, iters=64):
    """Quantile function F_1^{-1}(u) by vectorized bisection on the mixture CDF."""
    u = np.clip(np.asarray(u, dtype=float), U_CLIP, 1 - U_CLIP)
    q = mu + sigma * norm.ppf(u)[..., None]
    lo, hi = q.min(axis=-1), q.max(axis=-1)
    for _ in range(iters):
        mid = 0.5 * (lo + hi)
        below = mixture_cdf(mid, pi, mu, sigma) < u
        lo = np.where(below, mid, lo)
        hi = np.where(below, hi, mid)
    return 0.5 * (lo + hi)
```

## Sampling and evaluating a density

The transport sections built maps and fields that turn samples of a simple source into samples of a target. A generative model often has a second job. Given a point $x$, it should report how likely the model finds that point, the value $p_X(x)$ of its density. You need this number to train by maximum likelihood, to compare models on held-out data, and to flag inputs the model finds unlikely.

Sampling alone does not give you this number. If you push samples $z \sim p_Z$ through a function $G$, the outputs $x = G(z)$ follow some density $p_X$. To evaluate it at one point you have to know which $z$ produced $x$ and how much $G$ stretched the space around $z$. A normalizing flow is a generative model built so that both are easy to compute: $G$ is invertible, and the stretch has a formula.

## The change of variables formula

Let $Z$ have density $p_Z$ on $\mathbb{R}^d$ and let $G$ be an invertible, differentiable map of $\mathbb{R}^d$ onto itself. The density of $X = G(Z)$ is

$$
p_X(x) = p_Z\bigl(G^{-1}(x)\bigr)\,\bigl|\det J_{G^{-1}}(x)\bigr| ,
$$

where $J_{G^{-1}}$ is the Jacobian matrix of the inverse map. The determinant measures how much $G^{-1}$ scales a small volume around $x$. Mass is conserved: the mass in a small cell around $z$ ends up in the image of that cell, so if $G$ spreads the cell over a larger volume, the density falls by the same factor. In 1D the Jacobian is the derivative, and the formula reads $p_X(x) = p_Z(z)\,|dz/dx|$ with $z = G^{-1}(x)$.

The [transport maps and couplings section](https://aegean.ai/aiml-common/lectures/optimal-transport/maps-and-couplings/index.md) already built such a map. With $p_Z = \mathcal{N}(0, 1)$, CDF $\Phi$, and the mixture CDF $F_1$, the increasing map

$$
G = F_1^{-1} \circ \Phi, \qquad G^{-1} = \Phi^{-1} \circ F_1 ,
$$

sends standard normal samples to samples of the mixture. It is the map $T$ of the transport sections, with $z$ in place of $x_0$. It is increasing, so it is invertible, and it is a normalizing flow. The code below evaluates its change of variables density on a grid. It computes $G^{-1}$ from the mixture CDF and the derivative $dz/dx$ by central differences. The result is the mixture density $p_X = p_1$, with weights $(0.3, 0.5, 0.2)$, means $(-2, 1, 3.5)$ and standard deviations $(0.4, 0.6, 0.3)$. The code also confirms that $G^{-1}(G(z)) = z$.

```python theme={null}
def flow_forward(z):
    """G(z) = F_1^{-1}(Phi(z)): standard normal sample in, mixture sample out."""
    return mixture_ppf(norm.cdf(z))


def flow_inverse(x):
    """G^{-1}(x) = Phi^{-1}(F_1(x))."""
    return norm.ppf(mixture_cdf(x))


def flow_log_density(x, h=1e-5):
    """log p_X(x) = log p_Z(G^{-1}(x)) + log |dG^{-1}/dx|, the derivative by central differences."""
    z = flow_inverse(x)
    dz_dx = (flow_inverse(x + h) - flow_inverse(x - h)) / (2 * h)
    return norm.logpdf(z) + np.log(np.abs(dz_dx))


rng = np.random.default_rng(0)
x_grid = np.linspace(-3.5, 4.5, 801)
log_p_flow = flow_log_density(x_grid)
log_p_true = np.log(mixture_pdf(x_grid))
z_back = flow_inverse(flow_forward(x_grid / 2))
print(f"largest |log p_X from the flow - log mixture density| on [-3.5, 4.5]: "
      f"{np.max(np.abs(log_p_flow - log_p_true)):.1e}")
print(f"largest |G^{{-1}}(G(z)) - z| on [-1.75, 2.25]: {np.max(np.abs(z_back - x_grid / 2)):.1e}")
```

```output theme={null}
largest |log p_X from the flow - log mixture density| on [-3.5, 4.5]: 8.3e-09
largest |G^{-1}(G(z)) - z| on [-1.75, 2.25]: 4.0e-15
```

```python theme={null}
assert np.max(np.abs(log_p_flow - log_p_true)) < 1e-5
assert np.max(np.abs(z_back - x_grid / 2)) < 1e-8
```

![Output from cell 4](https://mintcdn.com/aegeanaiinc/CmbmEw2ZQMzm7U6e/aiml-common/lectures/normalizing-flows/images/cell_4_output_1.png?fit=max\&auto=format\&n=CmbmEw2ZQMzm7U6e\&q=85\&s=0cc93c25923342ab0aad61d4c561a4a4)

*Histogram of 200,000 samples $G(z)$, the mixture density (thick orange) and the change of variables density (thin blue). The two curves coincide.*

Sampling runs $G$ forward on a draw from $\mathcal{N}(0, 1)$. Evaluating runs $G^{-1}$ on the query point and corrects by the derivative. A neural flow uses the same two operations, with $G$ built from layers whose inverses and Jacobian determinants are cheap to compute.

## Composing maps

A single invertible layer is rarely flexible enough, so flows stack several. Write $G = G_K \circ \cdots \circ G_1$, start from $z_0 = z$, and set $z_k = G_k(z_{k-1})$, so that $x = z_K$. The determinant of a product of Jacobians is the product of their determinants, so the log-density picks up one term per layer:

$$
\log p_X(x) = \log p_Z(z) - \sum_{k=1}^{K} \log \bigl|\det J_{G_k}(z_{k-1})\bigr| .
$$

The terms add, so you can compute them layer by layer during the forward pass. The quantile map splits into two layers. The first, $G_1 = \Phi$, sends $\mathcal{N}(0, 1)$ to the uniform density on $(0, 1)$. The second, $G_2 = F_1^{-1}$, sends the uniform density to the mixture. The code below computes each log-derivative by central differences at five points and adds the terms.

```python theme={null}
def log_abs_derivative(f, x, h=1e-5):
    return np.log(np.abs((f(x + h) - f(x - h)) / (2 * h)))


z = norm.ppf([0.1, 0.3, 0.5, 0.7, 0.9])
u = norm.cdf(z)                         # first map:  G_1 = Phi, from N(0, 1) to uniform
x = mixture_ppf(u)                      # second map: G_2 = F_1^{-1}, from uniform to the mixture
logdet_1 = log_abs_derivative(norm.cdf, z)
logdet_2 = log_abs_derivative(mixture_ppf, u, h=1e-7)
log_p_chain = norm.logpdf(z) - logdet_1 - logdet_2
print("     z        x    log p_Z   -logdet_1   -logdet_2   sum     log p_X")
for row in zip(z, x, norm.logpdf(z), -logdet_1, -logdet_2, log_p_chain, np.log(mixture_pdf(x))):
    print("".join(f"{v:9.4f} " for v in row))
```

```output theme={null}
     z        x    log p_Z   -logdet_1   -logdet_2   sum     log p_X
  -1.2816   -2.1723   -1.7401    1.7401   -1.2994   -1.2994   -1.2994 
  -0.5244   -0.8375   -1.0564    1.0564   -4.9009   -4.9009   -4.9009 
   0.0000    0.8480   -0.9189    0.9189   -1.1334   -1.1334   -1.1334 
   0.5244    1.5050   -1.0564    1.0564   -1.4554   -1.4554   -1.4554 
   1.2816    3.5000   -1.7401    1.7401   -1.3242   -1.3242   -1.3242
```

```python theme={null}
assert np.allclose(log_p_chain, np.log(mixture_pdf(x)), atol=1e-5)
```

The first layer cancels $\log p_Z(z)$ exactly, because the uniform density equals 1 on $(0, 1)$. The second layer then carries the whole log-density of the mixture. The last two columns agree to the printed precision.

## Continuous flows

Stacking more and smaller layers leads to a flow in continuous time. A velocity field $v_t$ moves each particle by $\dot x_t = v_t(x_t)$, and the map from $x_0$ to $x_1$ is invertible because you can run the same equation backward in time. The log-determinant of a stack becomes an integral along each trajectory.

The integral follows from the continuity equation of the [dynamic transport section](https://aegean.ai/aiml-common/lectures/optimal-transport/dynamic-transport/index.md), $\partial_t p_t + \nabla \cdot (p_t v_t) = 0$. Expanding the divergence of the product gives

$$
\partial_t p_t + v_t \cdot \nabla p_t + p_t\, \nabla \cdot v_t = 0 .
$$

Along a trajectory $X_t$ the chain rule gives $\frac{d}{dt} p_t(X_t) = \partial_t p_t(X_t) + \nabla p_t(X_t) \cdot v_t(X_t)$, which is the sum of the first two terms. So $\frac{d}{dt} p_t(X_t) = -p_t(X_t)\,\nabla \cdot v_t(X_t)$. Dividing by $p_t(X_t)$ and integrating from $0$ to $1$,

$$
\begin{aligned}
\frac{d}{dt} \log p_t(X_t) &= -\nabla \cdot v_t(X_t), \\
\log p_1(X_1) &= \log p_0(X_0) - \int_0^1 \nabla \cdot v_t(X_t)\,dt .
\end{aligned}
$$

This is the instantaneous change of variables formula of continuous normalizing flows (Chen et al., 2018). The divergence is the trace of the Jacobian of $v_t$, so it replaces the determinant of the discrete formula. To evaluate $\log p_1(x)$ at a point, integrate the trajectory backward from $x$ to find $x_0$, and subtract the integral of the divergence along the way.

The displacement interpolation of the dynamic transport section gives a check with a known answer. Each particle moves at the constant velocity $T(x_0) - x_0$, and in 1D the divergence is $\partial_x v_t$. The code below tracks 16,001 particles at evenly spaced quantile levels. At each time it estimates $\partial_x v_t$ at every particle from the velocities and positions of its neighbors, and it integrates $-\partial_x v_t$ in time with the trapezoid rule. At $t = 1$ the result matches the log of the mixture density at $T(x_0)$.

```python theme={null}
T_GRID = np.linspace(0, 1, 1001)
LEVELS = np.linspace(1e-3, 1 - 1e-3, 16001)   # quantile levels of the tracked particles
X0 = norm.ppf(LEVELS)
VEL = flow_forward(X0) - X0                    # constant velocity T(x_0) - x_0 of each particle
PICK = np.searchsorted(LEVELS, [0.05, 0.25, 0.3, 0.45, 0.65, 0.85, 0.97])  # particles to plot

log_p = norm.logpdf(X0)                        # log p_0(x_0) at t = 0
history = [log_p[PICK]]
div_prev = np.gradient(VEL, X0)
for t_prev, t in zip(T_GRID[:-1], T_GRID[1:]):
    # Each particle keeps its velocity, so along the line of particles at time t the
    # divergence dv/dx is the finite difference of VEL over the positions x_t.
    div = np.gradient(VEL, X0 + t * VEL)
    log_p = log_p - 0.5 * (t - t_prev) * (div + div_prev)   # trapezoid rule in t
    div_prev = div
    history.append(log_p[PICK])
history = np.array(history)

err = np.abs(log_p - np.log(mixture_pdf(X0 + VEL)))
inner = (LEVELS > 0.01) & (LEVELS < 0.99)
print(f"largest |integrated log p_1 - log mixture density| for levels in (0.01, 0.99): "
      f"{err[inner].max():.1e}, at the particle that starts at x_0 = {X0[inner][np.argmax(err[inner])]:.2f}")
```

```output theme={null}
largest |integrated log p_1 - log mixture density| for levels in (0.01, 0.99): 2.6e-04, at the particle that starts at x_0 = -0.52
```

```python theme={null}
assert err[inner].max() < 1e-3
```

![Output from cell 9](https://mintcdn.com/aegeanaiinc/CmbmEw2ZQMzm7U6e/aiml-common/lectures/normalizing-flows/images/cell_9_output_1.png?fit=max\&auto=format\&n=CmbmEw2ZQMzm7U6e\&q=85\&s=47a6c68da77948c67952027fa9f8334c)

*Left, seven particle paths of the displacement interpolation. Right, the log-density along each path from the integrated divergence. The dots at $t = 1$ are the log of the mixture density at the end of each path.*

The largest error, printed above, is at the particle that starts near $x_0 = -0.52$ (green). It ends in the low-density gap between the first two components. Its neighbors head for different components, so the field is steep there at small $t$, and the finite differences in $x$ are least accurate. For a particle with $T'(x_0) > 0$ the integral has a closed form, $\int_0^1 \frac{T'(x_0) - 1}{1 - t + t\,T'(x_0)}\,dt = \log T'(x_0)$. The continuous formula therefore reproduces the change of variables formula for the map $T$.

## Flows, decoders and the models that follow

An invertible map keeps the dimension: $z$ and $x$ both live in $\mathbb{R}^d$. A latent-variable model such as the [variational autoencoder](https://aegean.ai/aiml-common/lectures/vae/introduction/index.md) decodes a low-dimensional $z$ into a high-dimensional $x$. Its decoder has no inverse, so the density of $x$ is an integral over all $z$ that could have produced it, and training works with a lower bound on that integral. A flow has an exact density, but its latent space has as many dimensions as the data.

The likelihood of a continuous flow is exact, but each evaluation integrates an ODE together with a divergence, and maximum likelihood training repeats that inside every gradient step. [Flow matching](https://aegean.ai/aiml-common/lectures/flow-matching/index.md) trains the field of a continuous normalizing flow by regression on a chosen probability path (Lipman et al., 2022). It never evaluates a likelihood during training. After training, the formula above still gives the likelihood of any point under the learned field. The probability flow ODE of [score-based diffusion](https://aegean.ai/aiml-common/lectures/diffusion/score-based/index.md) is a deterministic flow with the same marginals as the diffusion, and the same integral of the divergence evaluates likelihoods for a diffusion model (Song et al., 2020).

Key references: (Chen et al., 2018; Lipman et al., 2022; Song et al., 2020)

## References

* Chen, R., Rubanova, Y., Bettencourt, J., Duvenaud, D. (2018). *Neural ordinary differential equations*. arXiv \[cs.LG]. [DOI](https://doi.org/10.48550/arXiv.1806.07366)
* Lipman, Y., Chen, R., Ben-Hamu, H., Nickel, M., Le, M. (2022). *Flow Matching for generative modeling*. arXiv \[cs.LG]. [Link](http://arxiv.org/abs/2210.02747)
* Song, Y., Sohl-Dickstein, J., Kingma, D., Kumar, A., Ermon, S., et al. (2020). *Score-based generative modeling through stochastic differential equations*. arXiv \[cs.LG]. [DOI](https://doi.org/10.48550/arXiv.2011.13456)

***

Found an error on this page? [Report an issue](https://github.com/aegean-ai/eaia-feedback/issues/new?template=page-issue.yml\&page=/aiml-common/lectures/normalizing-flows).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.