Skip to main content
Open In Colab

Free energy as the selecting principle

The Fokker-Planck section measured the Ornstein-Uhlenbeck density path and found that it is not the geodesic between its endpoints. Its normalized action Aˉ\bar{\mathcal{A}} is almost four times W22(p0,pT)W_2^2(p_0, p_T), so least action does not select it. This section names the principle that does: the path descends a free energy as steeply as the Wasserstein geometry allows. You first see this on Gaussians, where the geometry is an ordinary plane and the descent can be drawn. The same statement then carries over to all densities on the line, where it can be solved one implicit step at a time. The running example is the one of the Fokker-Planck section: the three-component mixture p0p_0, the forward process dXt=−Xt dt+2 dWtdX_t = -X_t\,dt + \sqrt{2}\,dW_t, and the horizon T=4T = 4. The density ptp_t is a closed-form mixture, and the score is a responsibility-weighted sum of component scores. Free energies, actions and Fisher information come from quadrature, the Gaussian gradient flow from an ordinary differential equation solver, and the implicit steps from numerical optimization.

Gradient descent and its implicit step

Gradient flow moves a point zz downhill on a function EE with z˙=−∇E(z)\dot z = -\nabla E(z). Its implicit Euler step with time step τ\tau is zk+1=arg⁡min⁡z  E(z)+∥z−zk∥22τ.z_{k+1} = \arg\min_z \; E(z) + \frac{\lVert z - z_k \rVert^2}{2\tau} . Setting the gradient of the right side to zero gives (zk+1−zk)/τ=−∇E(zk+1)(z_{k+1} - z_k)/\tau = -\nabla E(z_{k+1}), the backward Euler update. The minimization uses only the function and the distance, so the step can be written on any space that has a distance, including a space of probability densities with the distance W2W_2.

Gaussians as a plane

For two Gaussians on the line the Wasserstein distance has a closed form, W22(N(m1,σ12),N(m2,σ22))=(m1−m2)2+(σ1−σ2)2.\begin{aligned} &W_2^2\bigl(\mathcal{N}(m_1, \sigma_1^2), \mathcal{N}(m_2, \sigma_2^2)\bigr) \\ &\quad = (m_1 - m_2)^2 + (\sigma_1 - \sigma_2)^2 . \end{aligned} The family of Gaussians is therefore the half plane of points (m,σ)(m, \sigma) with σ>0\sigma > 0, and W2W_2 is the ordinary Euclidean distance in that plane. Geodesics are straight segments. Take as free energy the KL divergence to the standard normal γ=N(0,1)\gamma = \mathcal{N}(0, 1). On Gaussians it is F(m,σ)=12(m2+σ2−1−2log⁡σ),\mathcal{F}(m, \sigma) = \tfrac12 \bigl(m^2 + \sigma^2 - 1 - 2 \log \sigma\bigr) , with its minimum 00 at (0,1)(0, 1). The gradient is (m,σ−1/σ)(m, \sigma - 1/\sigma), so the gradient flow in the plane is m˙=−m,σ˙=1σ−σ.\dot m = -m, \qquad \dot \sigma = \frac{1}{\sigma} - \sigma . Its solution is mt=m0e−tm_t = m_0 e^{-t} and σt=1+(σ02−1)e−2t\sigma_t = \sqrt{1 + (\sigma_0^2 - 1) e^{-2t}}. These are the mean and the standard deviation of the Ornstein-Uhlenbeck process started from N(m0,σ02)\mathcal{N}(m_0, \sigma_0^2), whose variance is σ02e−2t+1−e−2t\sigma_0^2 e^{-2t} + 1 - e^{-2t}. The process maps Gaussians to Gaussians, so on this family it is the gradient flow of the KL divergence in the plane. To check this, the gradient flow is integrated numerically from (m0,σ0)=(2.5,0.3)(m_0, \sigma_0) = (2.5, 0.3) over [0,T][0, T] and compared with the closed-form Ornstein-Uhlenbeck moments.
The two agree to the tolerance of the integrator. The plane also shows the difference between a gradient flow and a geodesic. Both paths start at z0=(m0,σ0)z_0 = (m_0, \sigma_0) and end at the Ornstein-Uhlenbeck point zT=(mT,σT)z_T = (m_T, \sigma_T). The geodesic is the straight segment traversed at constant speed, and its normalized action is ∥zT−z0∥2\lVert z_T - z_0 \rVert^2. Along the gradient flow, ddtF(zt)=∇F⋅z˙=−∥z˙∥2\frac{d}{dt}\mathcal{F}(z_t) = \nabla \mathcal{F} \cdot \dot z = -\lVert \dot z \rVert^2, so its action in physical time equals the drop F(z0)−F(zT)\mathcal{F}(z_0) - \mathcal{F}(z_T). The normalized action is TT times that, as in the Fokker-Planck section.
Contour plot of the free energy in the plane of mean and standard deviation, with closed level curves around the point (0, 1). A solid blue curve runs from the start at (2.5, 0.3) up and to the left, reaching a standard deviation near 1 by mean 1 and then running almost horizontally to the end point near (0.05, 1). A dashed orange straight line joins the same two points. Circle, square and triangle markers at times 0.5, 1 and 2 sit much farther along the blue curve than along the orange line. Level sets of the free energy in the plane of Gaussians. The solid curve is the gradient flow from (2.5,0.3)(2.5, 0.3), the dashed line the geodesic to the same endpoint. Markers of the same shape mark the same time on both paths. The gradient flow crosses each level set at a right angle, which is what steepest descent means in a Euclidean plane. At the start its direction is steeper than that of the geodesic: the standard deviation grows at 3.03.0 per unit time while the mean falls at 2.52.5. By t=1t = 1 the flow has σt=0.94\sigma_t = 0.94 and from then on it mostly moves the mean. The geodesic changes both at a constant rate. The flow has a lower KL at every printed time between the endpoints: 1.191.19 against 2.932.93 at t=0.5t = 0.5, and 0.0570.057 against 0.950.95 at t=2t = 2. It pays for that with a normalized action of 15.4915.49, more than twice the 6.516.51 of the straight line.

Free energy on densities

The free energy of a density pp on the line is the KL divergence to γ\gamma, F(p)=∫plog⁡p dx+∫x22 p dx+log⁡2π=DKL(p ∥ γ).\begin{aligned} \mathcal{F}(p) &= \int p \log p\,dx + \int \frac{x^2}{2}\,p\,dx + \log\sqrt{2\pi} \\ &= D_{\mathrm{KL}}(p \,\|\, \gamma) . \end{aligned} The first term is the negative entropy and the second a potential energy. The constant log⁡2π\log \sqrt{2\pi} has no effect on the flow. For a Gaussian, F\mathcal{F} reduces to the formula of the plane. In one dimension the quantile function Q=F−1Q = F^{-1} turns all three ingredients into simple integrals over u∈(0,1)u \in (0, 1). Substitute x=Q(u)x = Q(u). Then p(Q(u)) Q′(u)=1p(Q(u))\,Q'(u) = 1 and p dx=dup\,dx = du, so ∫plog⁡p dx=−∫01log⁡Q′(u) du,∫x22 p dx=∫01Q(u)22 du,W22(p,q)=∫01(Qp−Qq)2du.\begin{aligned} \int p \log p\,dx &= -\int_0^1 \log Q'(u)\,du , \\ \int \frac{x^2}{2}\,p\,dx &= \int_0^1 \frac{Q(u)^2}{2}\,du , \\ W_2^2(p, q) &= \int_0^1 \bigl(Q_p - Q_q\bigr)^2 du . \end{aligned} The last line says that W2W_2 is the ordinary L2L^2 distance between quantile functions. A Wasserstein gradient flow on the line is therefore an ordinary L2L^2 gradient flow of F\mathcal{F}, written as a functional of QQ on the set of increasing functions (Santambrogio, 2015). The plane of Gaussians is the special case Q(u)=m+σ Φ−1(u)Q(u) = m + \sigma\,\Phi^{-1}(u), with Φ\Phi the standard normal CDF. The same flow can be written on the density. The first variation of F\mathcal{F} is log⁡p+x2/2+1\log p + x^2/2 + 1. A Wasserstein gradient flow moves mass with the velocity v=−∂xv = -\partial_x of the first variation, so vt(x)=−∂x(log⁡pt+x22)=−x−∂xlog⁡pt.v_t(x) = -\partial_x\Bigl(\log p_t + \frac{x^2}{2}\Bigr) = -x - \partial_x \log p_t . This is the probability flow velocity of the Fokker-Planck section. The continuity equation ∂tpt+∂x(ptvt)=0\partial_t p_t + \partial_x(p_t v_t) = 0 with this velocity is ∂tpt=∂x(x pt)+∂x2pt\partial_t p_t = \partial_x(x\,p_t) + \partial_x^2 p_t, the Fokker-Planck equation of the Ornstein-Uhlenbeck process. The density path of diffusion is the Wasserstein gradient flow of the KL divergence to its equilibrium. All quantile computations on this page use one grid of levels. The levels are ui=Φ(zi)u_i = \Phi(z_i) with ziz_i equally spaced, which puts more levels in the tails. On that grid, Q′Q' is the finite difference on each interval and the integrals use the trapezoid rule.
At t=0t = 0 the quantile form of F\mathcal{F} is about 5×10−35 \times 10^{-3} below the integral in xx, because the narrow components and the gaps between them are hardest to resolve. The error falls when the grid is refined (see the numerical checks).

Dissipation equals action

Along the flow the free energy falls at a rate fixed by the velocity. Differentiate F(pt)\mathcal{F}(p_t) in time, substitute the continuity equation, and integrate by parts: ddtF(pt)=∫∂tpt(log⁡pt+x22+1)dx=∫pt vt ∂x(log⁡pt+x22)dx=−∫pt vt2 dx.\begin{aligned} \frac{d}{dt}\mathcal{F}(p_t) &= \int \partial_t p_t \Bigl(\log p_t + \frac{x^2}{2} + 1\Bigr) dx \\ &= \int p_t\,v_t\,\partial_x\Bigl(\log p_t + \frac{x^2}{2}\Bigr) dx \\ &= -\int p_t\,v_t^2\,dx . \end{aligned} Since vt=−∂xlog⁡(pt/γ)v_t = -\partial_x \log(p_t/\gamma), the last integral is the relative Fisher information I(pt ∥ γ)=∫pt ∣∂xlog⁡(pt/γ)∣2 dxI(p_t \,\|\, \gamma) = \int p_t\,\lvert \partial_x \log(p_t/\gamma) \rvert^2\,dx. Integrating from 00 to TT gives F(p0)−F(pT)=∫0T ⁣ ⁣∫pt vt2 dx dt=A,\mathcal{F}(p_0) - \mathcal{F}(p_T) = \int_0^T \!\! \int p_t\,v_t^2\,dx\,dt = \mathcal{A} , the physical-time action of the Fokker-Planck section. The Benamou-Brenier formula bounds W22W_2^2 by the normalized action Aˉ=TA\bar{\mathcal{A}} = T\mathcal{A} (Benamou & Brenier, 2000), so W22(p0,pT)≤T [F(p0)−F(pT)].W_2^2(p_0, p_T) \le T\,\bigl[\mathcal{F}(p_0) - \mathcal{F}(p_T)\bigr] . Numerically, F(pt)\mathcal{F}(p_t) comes from quadrature on [−12,12][-12, 12] with log⁡pt\log p_t computed by log-sum-exp. Its time derivative comes from central differences with step 10−410^{-4}, at 41 times in [0.01,T][0.01, T], and is compared with the Fisher information computed from the exact velocity. The action integrates the Fisher information over 801 times, spaced more densely near t=0t = 0.
The rate of change of F\mathcal{F} matches the Fisher information at every time, and the drop F(p0)−F(pT)=1.4484\mathcal{F}(p_0) - \mathcal{F}(p_T) = 1.4484 matches the action to five digits. The Fokker-Planck section reported A=1.4454\mathcal{A} = 1.4454 and W22=1.4998W_2^2 = 1.4998 because it restricted the uu integrals to [0.001,0.999][0.001, 0.999]. On that same range the computation here gives the same two numbers. Over the whole line the tails add 0.0030.003 to A\mathcal{A} and 0.0010.001 to W22W_2^2. The bound holds with room to spare: 1.50≤5.791.50 \le 5.79. The difference between the two sides is the excess action Aˉ−W22\bar{\mathcal{A}} - W_2^2 of the Fokker-Planck section.

The JKO step

The implicit step of the first table, written with F\mathcal{F} and W2W_2, is the JKO step (Jordan et al., 1998): pk+1=arg⁡min⁡p  F(p)+W22(p,pk)2τ.p_{k+1} = \arg\min_p \; \mathcal{F}(p) + \frac{W_2^2(p, p_k)}{2\tau} . Jordan, Kinderlehrer and Otto proved that as τ→0\tau \to 0 these steps converge to the solution of the Fokker-Planck equation. Each step trades a lower free energy against the cost of moving mass, without using a velocity or a score. On the line the step is a minimization over quantile functions, Qk+1=arg⁡min⁡Q  −∫01log⁡Q′ du+∫01Q22 du+12τ∫01(Q−Qk)2 du.\begin{aligned} Q_{k+1} = \arg\min_Q \; & -\int_0^1 \log Q'\,du + \int_0^1 \frac{Q^2}{2}\,du \\ & + \frac{1}{2\tau} \int_0^1 (Q - Q_k)^2\,du . \end{aligned} Every experiment uses the same discretization on the grid above. The unknowns are Q(u0)Q(u_0) and the logarithms of the spacings Q(ui+1)−Q(ui)Q(u_{i+1}) - Q(u_i), so the computed QQ is increasing by construction. In these variables the entropy term is linear. The minimization uses L-BFGS with the exact gradient, warm-started from the previous step. The error of step kk is the W2W_2 distance on the grid between QkQ_k and the exact quantile function at time kτk\tau. The first test starts from the Gaussian (m0,σ0)=(2.5,0.3)(m_0, \sigma_0) = (2.5, 0.3), where the exact answer is (mt,σt)(m_t, \sigma_t) from the plane. It runs to TT with τ=0.1\tau = 0.1 and τ=0.05\tau = 0.05.
Halving τ\tau halves the largest error, from 0.0510.051 to 0.0270.027. This is the first-order accuracy of an implicit Euler step. At both time steps the largest error occurs within the first half unit of time, where the narrow start widens fastest. The same steps then run from the mixture p0p_0, and each step is compared with the exact pkτp_{k\tau}.
Density against x at times 0, 0.2, 0.5, 1 and 2. Solid lines are the exact densities: three peaks at time 0 that merge into one broad bump by time 2. Dots from the JKO steps lie on the lines at every time, with the largest visible gap at time 0.2 between x = -1 and x = 0, where the dots sit below the line. JKO steps at t=0,0.2,0.5,1,2t = 0, 0.2, 0.5, 1, 2 with τ=0.05\tau = 0.05 from the mixture (dots, the density Δu/ΔQ\Delta u / \Delta Q of each step) and the exact Ornstein-Uhlenbeck density at the same time (lines). The largest W2W_2 error over all steps is 0.0300.030 at τ=0.1\tau = 0.1 and 0.0160.016 at τ=0.05\tau = 0.05. The steps differ most from the exact density in the gap between the left and middle components, which fills quickly at the start. At x=−0.5x = -0.5 and t=0.2t = 0.2 the exact density is 0.0990.099, against 0.0800.080 for τ=0.1\tau = 0.1 and 0.0880.088 for τ=0.05\tau = 0.05. The steps lag behind the exact path there, as implicit Euler steps do, and the lag shrinks roughly in proportion to τ\tau; doubling the grid leaves it unchanged. To within these errors, a sequence of minimizations that uses only F\mathcal{F} and W2W_2 reproduces the diffusion density path.

Geodesic or gradient flow

The plane of Gaussians showed two paths between the same endpoints. The same comparison works for the mixture. Over t∈[0,T]t \in [0, T], the Ornstein-Uhlenbeck marginals run beside the constant-speed displacement interpolation from p0p_0 to pTp_T, whose quantile function is (1−t/T) Q0+(t/T) QT(1 - t/T)\,Q_0 + (t/T)\,Q_T. Along the geodesic the KL comes from the quantile form of F\mathcal{F}, since the interpolation is defined through its quantile function.
Two standard transport heatmaps with time from 0 to 4 on the horizontal axis. On the left, the gradient flow: the three bands of p_0 merge into one band before t = 1.5, and the quantile trajectories bend sharply at the start and are flat afterwards. Its title reports kinetic action 5.793. On the right, the geodesic: the trajectories are straight lines, the three bands stay apart until close to t = 4, and the title reports kinetic action 1.501. Left, the Ornstein-Uhlenbeck path, which is the gradient flow, with quantile trajectories at 25 levels. Right, the geodesic between the same p0p_0 and pTp_T, with straight trajectories. Each title reports the normalized action: TAT\mathcal{A} on the left and W22W_2^2 on the right. KL divergence to the standard normal against time on a log scale. The solid gradient flow curve falls from about 1.4 at t = 0 along a nearly straight line to about 6e-5 at t = 4. The dashed geodesic curve stays above it, falling slowly to about 0.15 at t = 3 and then dropping steeply to the same final value at t = 4. KL divergence to N(0,1)\mathcal{N}(0, 1) against time along the gradient flow (solid) and the geodesic (dashed), on a log scale. The two paths share their endpoints and differ everywhere in between. The gradient flow has the lower KL at every one of the 79 interior times: 0.260.26 against 1.161.16 at t=0.5t = 0.5, and 0.0040.004 against 0.470.47 at t=2t = 2. The local decay rate −ddtlog⁡DKL-\frac{d}{dt}\log D_{\mathrm{KL}} of the flow falls from about 3.03.0 at t=1t = 1 to 2.12.1 at t=3t = 3, so the late decay is close to e−2te^{-2t}. The geodesic loses its KL more steadily: from 1.451.45 to 0.470.47 at t=2t = 2 and 0.1550.155 at t=3t = 3, about 90%90\% of it by 3T/43T/4. It looks slow on the log scale because the axis spans down to about 6×10−56 \times 10^{-5}. Over the last unit of time it drops from 0.1550.155 to 6×10−56 \times 10^{-5}. The geodesic has the smaller normalized action, 1.501.50 against 5.795.79. Each path is optimal for a different problem. A geodesic joins two prescribed densities with the least action, and it needs both of them to be defined. A gradient flow needs only a starting density and a free energy. It follows the steepest descent of F\mathcal{F}, and the endpoint is wherever the flow has reached at time TT. This flow approaches γ\gamma only as t→∞t \to \infty and never reaches it in finite time. For that reason the Fokker-Planck section used the true endpoint pTp_T throughout.

Back to the three levels

The Fokker-Planck section opened with three levels of description. Both paths of this section can now be placed on all three. Once the principle is fixed, so is the density path: the gradient flow from p0p_0 has one solution, and so does the geodesic from p0p_0 to pTp_T. The density path does not fix the particles, as the family of dynamics with shared marginals showed in the Fokker-Planck section. Diffusion and the geodesic differ in the principle, and that difference accounts for the factor of almost four between their actions.

Numerical checks

The JKO results depend on three numerical choices. Each one is varied separately, for the mixture with τ=0.05\tau = 0.05. The optimizer tolerance changes from 10−1210^{-12} to 10−910^{-9}. The quantile grid doubles from NUN_U to 2NU2N_U levels. The time step takes both values of τ\tau, for both starting densities.
The time step dominates the error. Halving τ\tau lowers the largest W2W_2 error by about 0.0140.014 for the mixture and 0.0240.024 for the Gaussian, close to a factor of two in both cases. Loosening the optimizer tolerance by three orders of magnitude moves the iterates by about 10−310^{-3} in W2W_2 and changes the largest error by less than 10−410^{-4}. Tightening it to 10−1410^{-14}, near the limit of double precision, moves them by about 10−410^{-4}, so the reference iterates are converged far below the time-step error. Doubling the grid changes the largest error by 2×10−42 \times 10^{-4}, leaves the density in the gap unchanged, and brings the quantile-form free energy of p0p_0 from 5.5×10−35.5 \times 10^{-3} to 1.1×10−31.1 \times 10^{-3} below quadrature in xx.

References

  • Benamou, J., Brenier, Y. (2000). A computational fluid mechanics solution to the Monge-Kantorovich mass transfer problem. Numer. Math. (Heidelb.). DOI
  • Jordan, R., Kinderlehrer, D., Otto, F. (1998). The variational formulation of the Fokker—Planck equation. SIAM J. Math. Anal. DOI
  • Santambrogio, F. (2015). Optimal transport for applied mathematicians – calculus of variations, PDEs and modeling.

Found an error on this page? Report an issue.