3. DDPM: diffusion in discrete time
Draft
The chain and its schedules
DDPM makes diffusion concrete as a sequence of Gaussian steps. The forward chain adds noise over \(T\) steps; a learned reverse chain removes it over \(T\) steps. Training fits the reverse chain through a bound on the likelihood. Ho, Jain and Abbeel developed this approach following the work of Sohl-Dickstein, Weiss, Maheswaranathan and Ganguli.J. Sohl-Dickstein, E. A. Weiss, N. Maheswaranathan and S. Ganguli, "Deep unsupervised learning using nonequilibrium thermodynamics", ICML 2015 (arXiv 1503.03585): "systematically and slowly destroy structure in a data distribution through an iterative forward diffusion process. We then learn a reverse diffusion process that restores structure in data". J. Ho, A. Jain and P. Abbeel, "Denoising diffusion probabilistic models", NeurIPS 2020 (arXiv 2006.11239): the \(\varepsilon\)-parameterization, the simplified loss, \(T = 1000\), and "a state-of-the-art FID score of 3.17" on CIFAR-10, against the training set; against the test set the paper reports 5.24.
We already have the forward kernel from Section 1. Now we will use it on a time grid, derive the reverse conditional, and turn the likelihood bound into a regression loss. The examples on R2 let us vary the number of steps and see where the approximation breaks down.
The chain breaks generation into local prediction problems. Each forward step weakens the current signal a little and adds a little fresh noise. Reversal asks for a distribution of plausible previous states. With sufficiently small steps, this local distribution is easier to approximate than the entire distribution of clean data given almost pure noise.
left the law of the previous state given the current one, q(x_{i-1} | x_i), on R2 at the last step
of DDPM's chain of 1,000 steps, for every current state x_i (horizontal) and previous state
(vertical): a thin ridge along the rescaled diagonal; at the probe x_T = 0.5 it is a bell of
width 0.141 at 0.495, below 1e-12 nats from its own Gaussian and from the sampler's
right the law of the clean point given the same x_i, q(x_0 | x_i): three horizontal bands, R2
itself, since x_T keeps 0.0064 x_0; at the probe its mean is −0.236 and its width 1.186, and
it is 0.651 nats from any Gaussian (R2 itself: 0.651), 0.962 of its variance between the bumps
(the live figure needs JavaScript)
Definition 3.1.1 (the forward chain). Given rates \(\beta_1, \dots, \beta_T \in (0, 1)\), the forward chain is \(x_0 \sim p_0\) and
\[q(x_i \mid x_{i-1}) = \mathcal N\bigl(x_i;\, \sqrt{1 - \beta_i}\, x_{i-1},\; \beta_i I\bigr), \qquad i = 1, \dots, T .\]Write \(\alpha_i = 1 - \beta_i\) and \(\bar\alpha_i = \prod_{j \le i} \alpha_j\).
Proposition 3.1.2 (the chain in closed form). For every \(i\), \(q(x_i \mid x_0) = \mathcal N(x_i; \sqrt{\bar\alpha_i}\, x_0, (1 - \bar\alpha_i) I)\), so that \(x_i = \sqrt{\bar\alpha_i}\, x_0 + \sqrt{1 - \bar\alpha_i}\, \varepsilon\) with \(\varepsilon \sim \mathcal N(0, I)\); the chain is the forward process of Section 1 on the variance-preserving schedule \(\alpha_{t_i} = \sqrt{\bar\alpha_i}\), \(\sigma_{t_i} = \sqrt{1 - \bar\alpha_i}\), sampled at \(T\) times.
Proof. By induction with Proposition 1.1.8: the step kernel has \(\alpha_{i \mid i-1} = \sqrt{\alpha_i}\) and \(\sigma_{i \mid i-1}^2 = \beta_i = 1 - \alpha_i\), and composing it with \(\mathcal N(\sqrt{\bar\alpha_{i-1}}\, x_0, (1 - \bar\alpha_{i-1}) I)\) gives mean \(\sqrt{\alpha_i \bar\alpha_{i-1}}\, x_0 = \sqrt{\bar\alpha_i}\, x_0\) and variance \(\alpha_i (1 - \bar\alpha_{i-1}) + 1 - \alpha_i = 1 - \bar\alpha_i\). ∎
The product \(\bar\alpha_i\) records how much signal survives all the steps so far. Each step multiplies the signal, so these factors multiply rather than add. There is a notation trap: DDPM's \(\alpha_i = 1-\beta_i\) is a one-step variance factor; the signal coefficient called \(\alpha_t\) in Sections 1 and 2 is \(\sqrt{\bar\alpha_i}\) at this grid point.
Example 3.1.3 (DDPM's schedule). Ho et al. take \(T = 1000\) and \(\beta_i\) increasing linearly from \(10^{-4}\) to \(0.02\). Then \(\bar\alpha_T = 4.04 \times 10^{-5}\): the last state is \(0.0064\, x_0\) plus noise of variance \(0.99996\), and the prior mismatch of Remark 1.4.5, \(\mathrm{KL}(q(x_T \mid x_0) \,\|\, \mathcal N(0, I))\), is of the order \(\bar\alpha_T \|x_0\|^2/2\), a few times \(10^{-5}\) per dimension for data scaled to \([-1, 1]\). Halfway, \(\bar\alpha_{500} = 0.0786\).
The worked examples of this series (Section 11) keep the same \(\beta_i\) and, beside DDPM's \(T = 1000\), take \(T = 150\), the 150-step chain of what follows: then \(\sum_i \beta_i = 1.5075\) rather than \(10.05\), \(\bar\alpha_T = 0.2192\), and the last state is \(0.468\, x_0\) plus noise of standard deviation \(0.884\), with \(\lambda_T = -1.27\) rather than \(-10\); Example 3.1.5 measures what that costs. The figure below draws both chains with its chain control, and its third view the per-step weight of the bound of Section 3.3, which on the 150-step chain is \(1.168\) at the first step and \(0.0131\) at the last.
Nichol and Dhariwal observed that the linear schedule spends many steps at the end on nearly pure noise and proposed the cosine schedule, \(\bar\alpha_i = f(i/T)/f(0)\) with \(f(u) = \cos^2\bigl(\tfrac{u + s}{1 + s} \cdot \tfrac\pi2\bigr)\) and \(s = 0.008\), whose \(\beta_i = 1 - \bar\alpha_i/\bar\alpha_{i-1}\) they clip at \(0.999\).A. Nichol and P. Dhariwal, "Improved denoising diffusion probabilistic models", ICML 2021 (arXiv 2102.09672), Section 3.2, Equation 17; "we selected \(s\) such that \(\sqrt{\beta_0}\) was slightly smaller than the pixel bin size \(1/127.5\), which gives \(s = 0.008\)". The same paper learns the reverse variances, which the sampler of Section 3.3 takes as fixed, and reports that this "allows sampling with an order of magnitude fewer forward passes with a negligible difference in sample quality". On the clock \(t = i/T\) the cosine schedule has \(\alpha_t = \cos(\tfrac{t + s}{1 + s}\tfrac\pi2)/\cos(\tfrac{s}{1 + s}\tfrac\pi2)\) and a log signal-to-noise ratio of \(3.55\) at \(t = 0.1\), \(-0.02\) at \(t = 0.5\) and \(-3.70\) at \(t = 0.9\), against \(-2.46\) at \(t = 0.5\) for the linear schedule.
left alpha_t (solid) and sigma_t (dashed) for the linear-beta, cosine, linear-path and variance-
exploding schedules; dots: DDPM's chain of 1,000 steps, or the 150-step chain with the same betas
right the log signal-to-noise ratio of each with the chain's steps as a rug, or the time each
spends per unit of it, or the per-step weight of the evidence lower bound against L_simple
(the live figure needs JavaScript)
Proposition 3.1.4 (the continuous limit). Let \(\beta_i = \bar\beta(i/T)/T\) for a continuous rate \(\bar\beta\) on \([0, 1]\). As \(T \to \infty\), the chain of Definition 3.1.1 at the times \(t_i = i/T\) converges to the variance-preserving process \(dx = -\tfrac12\bar\beta_t x\, dt + \sqrt{\bar\beta_t}\, dW\) of Example 1.1.6, and \(\bar\alpha_{\lfloor tT \rfloor} \to e^{-B(t)}\) with \(B(t) = \int_0^t \bar\beta\).
Proof. \(\log\bar\alpha_i = \sum_{j \le i}\log(1 - \bar\beta(j/T)/T) = -\tfrac1T\sum_{j \le i}\bar\beta(j/T) + O(1/T)\), a Riemann sum for \(-B(t)\). The step, with \(\Delta t = 1/T\),
\[\begin{aligned} x_i - x_{i-1} &= (\sqrt{1 - \beta_i} - 1)\, x_{i-1} + \sqrt{\beta_i}\, \varepsilon_i \\ &= -\tfrac12\bar\beta(t_i)\, x_{i-1}\, \Delta t + \sqrt{\bar\beta(t_i)\, \Delta t}\, \varepsilon_i + O(\Delta t^2), \end{aligned}\]is the Euler–Maruyama step of the stated equation. ∎Song et al. (2021), Section 3.4 and Appendix B; with DDPM's \(\beta_i\) one has \(\bar\beta(t) = 0.1 + 19.9\, t\), so \(B(1) = 10.05\) and \(e^{-B(1)} = 4.3 \times 10^{-5}\), against the chain's \(\bar\alpha_{1000} = 4.04 \times 10^{-5}\); the difference is the second-order term of the logarithm.
Example 3.1.5 (the 150-step chain, in practice). We compare two models on FashionMNIST at \(28 \times 28\), with 60,000 training images scaled to \([-1, 1]\) and 10,000 test images held out. The first is the \(\varepsilon\)-model of Section 11, a small U-Net \(\varepsilon_\theta\) of widths (64, 64, 128), trained for forty epochs on the 150-step chain, \(\beta_i\) linear from \(10^{-4}\) to \(0.02\). The second uses the same architecture on DDPM's chain of 1,000 steps.
We sample both models by DDIM with \(\eta = 0\) (Section 5.1) and by the ancestral chain re-spaced to \(K\) steps, from 1,024 fixed draws of \(\mathcal N(0, I)\), and measure the Fréchet distance of the 1,024 samples to 1,024 held-out images in the 128-dimensional features of a small classifier trained on the same training set (FD_clf, comparable only with itself and not with FID).
The 150-step chain keeps DDPM's \(\beta_i\) and stops at \(T = 150\), so it ends at \(\bar\alpha_T = 0.2192\): the last state is \(0.468\, x_0\) plus noise of standard deviation \(0.884\), \(\lambda_T = -1.27\), and on FashionMNIST at \(28 \times 28\) the relative entropy of \(q(x_T \mid x_0)\) from \(\mathcal N(0, I)\) is \(70.05\) nats per image on average over the held-out images, \(0.089\) per pixel, against \(0.0109\) for DDPM's chain of 1,000 steps. Algorithm 3.3.2 starts from \(\mathcal N(0, I)\) nonetheless. The figure measures the cost of this mismatch.
To calibrate this distance, held-out images noised to index 15 of the chain score \(64.7\) on it. On that scale the model's DDIM samples from \(\mathcal N(0, I)\) are at Fréchet distance \(731.3\) from the held-out images, whatever the number of steps beyond five, and the fresh noise of the ancestral chain brings them to \(586.7\) at the full chain. Starting the same model from the true \(q(x_{149} \mid x_0)\) of held-out images leaks information about the image, so this is a diagnostic rather than a usable sampler. It reaches \(20.2\) in twenty-five steps, close to the \(11.7\) that two halves of the held-out set are apart.
The same architecture trained on DDPM's chain of 1,000 steps scores \(217.7\) from \(\mathcal N(0, I)\) and \(238.1\) from the true start: no improvement from removing the starting mismatch here. The remaining distance from the floor reflects the model's limitations. The model of the 150-step chain is being asked, at its first step, about an input it never saw in training; Example 5.4.2 measures what it makes of it at every number of steps, and Example 3.3.4 films its twin, whose chain does reach noise, step by step. ◁
top one held-out FashionMNIST image at sixteen indices of the 150-step chain (beta linear from
1e-4 to 0.02), noised by the closed form of Proposition 3.1.2; at the last index the image
keeps 47 % of its signal; under each state the change map x_i - eps, the state minus the
N(0, I) draw it was made with, which at the last index is the true start minus the sampler's
middle the log signal-to-noise ratio of the 150-step chain and of DDPM's chain of 1,000 steps, and
the relative entropy of q(x_i | x_0) from N(0, I) per image, 70.1 nats at the end of the
150-step chain against 0.011 at the end of DDPM's; beside it the state at the chosen index,
its draw and their difference, enlarged
bottom sixteen samples of the epsilon-model at K steps from N(0, I), or from the true
q(x_{T-1} | x_0) of held-out images, and the Frechet distance FD_clf against K for both chains
(the live figure needs JavaScript and loads 0.86 MB of data)
The reverse conditional
To sample backward, we would like to draw from \(q(x_{i-1} \mid x_i)\): the previous state given the current one. The reverse chain is Markov, but this transition depends on the unknown data law \(p_0\) and is generally not Gaussian.
Conditioning on the clean point changes the problem. Once \(x_0\) is known, the reverse transition is Gaussian, with a mean and variance we can calculate. That tractable conditional gives us a target for training the reverse chain.
With both endpoints known, there are two clues to the previous state. The clean point predicts where the forward chain should have reached; the current noisy point constrains the last step it could have taken. Multiplying their Gaussian densities combines these clues. The more precise clue receives more weight, and the remaining variance records the uncertainty left after using both.
left the step of Example 3.2.2 (T = 10, the last step, x_i = −1.1374) as Bayes' rule: on the axis of
the previous state, the clean point's clue, a bell of width 0.279 at −1.271; the current
point's clue, a bell of width 0.143 at −1.149; and their product, a bell of width 0.127 at
−1.174, 0.21 of the way from the current clue toward the clean one (precisions 12.9 and 49.0)
right the clean clue's share of the precision, beta_i / (1 − abar_i), against i/T for the chains of
10, 150 and 1,000 steps: 1 at the first step, 0.545 at the second step of DDPM's chain, below
0.1 from its step 17 and 0.020 at its last
(the live figure needs JavaScript)
Proposition 3.2.1 (the posterior of a step). For \(i \ge 2\),
\[\begin{aligned} q(x_{i-1} \mid x_i, x_0) &= \mathcal N\bigl(x_{i-1};\, \tilde\mu_i(x_i, x_0),\, \tilde\beta_i I\bigr), \\ \tilde\mu_i &= \frac{\sqrt{\bar\alpha_{i-1}}\, \beta_i}{1 - \bar\alpha_i}\, x_0 + \frac{\sqrt{\alpha_i}\,(1 - \bar\alpha_{i-1})}{1 - \bar\alpha_i}\, x_i, \\ \tilde\beta_i &= \frac{1 - \bar\alpha_{i-1}}{1 - \bar\alpha_i}\, \beta_i . \end{aligned}\]In the notation of Proposition 1.1.8, with \(s = t_{i-1}\) and \(t = t_i\), the mean is \(\frac{\alpha_{t \mid s}\sigma_s^2}{\sigma_t^2}\, x_t + \frac{\alpha_s \sigma_{t \mid s}^2}{\sigma_t^2}\, x_0\) and the variance is \(\sigma_{s \mid t}^2 = \sigma_{t \mid s}^2 \sigma_s^2/\sigma_t^2\).
Proof. By Bayes' rule, \(q(x_{i-1} \mid x_i, x_0) \propto q(x_i \mid x_{i-1})\, q(x_{i-1} \mid x_0)\), the first factor by the Markov property. Both factors are Gaussian in \(x_{i-1}\), with precisions \(\alpha_i/\beta_i\) and \(1/(1 - \bar\alpha_{i-1})\) and means \(x_i/\sqrt{\alpha_i}\) and \(\sqrt{\bar\alpha_{i-1}}\, x_0\). A product of Gaussians is Gaussian with the sum of the precisions and the precision-weighted mean: \(1/\tilde\beta_i = \alpha_i/\beta_i + 1/(1 - \bar\alpha_{i-1}) = (1 - \bar\alpha_i)/(\beta_i(1 - \bar\alpha_{i-1}))\), using \(\alpha_i(1 - \bar\alpha_{i-1}) + \beta_i = 1 - \bar\alpha_i\), and \(\tilde\mu_i = \tilde\beta_i\bigl(\sqrt{\alpha_i}\, x_i/\beta_i + \sqrt{\bar\alpha_{i-1}}\, x_0/(1 - \bar\alpha_{i-1})\bigr)\), which simplifies to the stated form. The second form is the same computation with \(\alpha_{t \mid s}^2 = \alpha_i\), \(\sigma_{t \mid s}^2 = \beta_i\), \(\sigma_s^2 = 1 - \bar\alpha_{i-1}\) and \(\sigma_t^2 = 1 - \bar\alpha_i\), and it holds for any schedule, not only a variance-preserving one. ∎
The posterior mean interpolates between the current point and the clean one, and the clean one is unknown at sampling time. The sampler replaces it by its best estimate, \(\hat x_0 = D_\theta(x_i)\); in the \(\varepsilon\) form of Ho et al., with \(\hat x_0 = (x_i - \sqrt{1 - \bar\alpha_i}\, \varepsilon_\theta(x_i, i))/\sqrt{\bar\alpha_i}\), the mean becomes
\[\mu_\theta(x_i, i) \;=\; \frac{1}{\sqrt{\alpha_i}}\Bigl(x_i - \frac{\beta_i}{\sqrt{1 - \bar\alpha_i}}\, \varepsilon_\theta(x_i, i)\Bigr). \tag{3.1}\]Replacing \(x_0\) by its conditional mean gives the correct mean of the reverse step when the denoiser is exact. It does not give the whole reverse distribution. Different possible clean sources contribute different Gaussian components; averaging their means leaves out uncertainty between those components. This is why an exact noise predictor can still produce an approximate finite-step DDPM sampler. The small-step limit and the choice of reverse variance matter.
Example 3.2.2 (one step on R2). The figure below runs the chain on R2 with \(T = 10\) and inspects, for one particle, the first step from pure noise, \(i = 9\) in the figure's count from zero, where \(\bar\alpha_i = 0.904\), \(\beta_i = 0.02\), \(\sqrt{\bar\alpha_i} = 0.951\) and \(\sqrt{1 - \bar\alpha_i} = 0.310\). The particle sits at \(x_i = -1.1374\); the exact noise estimate there is \(\hat\varepsilon = 0.391\), hence \(\hat x_0 = (-1.1374 - 0.310 \times 0.391)/0.951 = -1.324\), the posterior mean \(\mu_\theta = -1.174\), the standard deviation \(\sqrt{\tilde\beta_i} = 0.127\) added, and the particle lands at \(-1.339\).
The denoised estimate moves nearly all the way to the leftmost bump of R2 in one step, because at \(T = 10\) the first step already carries \(\bar\alpha_i\) from \(1\) to \(0.904\) and the chain never visits the regime of large noise at all: the chain's first state has \(\bar\alpha_T = 0.904\), not \(4 \times 10^{-5}\), when \(T = 10\) and the \(\beta_i\) keep their values. This is the coarseness the figure's \(W_1\) measures. A third choice, which the figure offers and which the sampler of Example 3.3.4 uses, keeps the same mean and adds noise of variance \(\beta_{i-1}\), the rate of the step before; at \(i = 149\) of the 150-step chain \(\beta_i\), \(\tilde\beta_i\) and \(\beta_{i-1}\) are \(0.02000\), \(0.019885\) and \(0.019866\). ◁
The evidence lower bound and the sampler
The learned chain starts from \(p_\theta(x_T) = \mathcal N(0, I)\) and uses Gaussian reverse steps, \(p_\theta(x_{i-1} \mid x_i) = \mathcal N(x_{i-1}; \mu_\theta(x_i, i), \Sigma_i)\), with \(\Sigma_i = \tilde\beta_i I\) or \(\beta_i I\). Evaluating a data point's likelihood would require integrating over all the intermediate states. The evidence lower bound avoids that calculation by comparing the learned transitions with the tractable conditionals we just derived.
Read the bound as a sum of three costs: matching the noisy starting law, matching each reverse transition, and reconstructing the data at the final step. The middle costs are tractable because training supplies the clean point, so both distributions being compared are known Gaussians apart from the network's prediction. Matching their means becomes a squared-error loss. The name refers to a lower bound on log likelihood; with the minus sign used below, it is an upper bound on negative log likelihood.
Theorem 3.3.1 (the evidence lower bound). For every \(x_0\),
\[\begin{aligned} -\log p_\theta(x_0) \;\le\; \mathbb E_q\Bigl[\, &\mathrm{KL}\bigl(q(x_T \mid x_0) \,\|\, p_\theta(x_T)\bigr) \\ &+ \sum_{i = 2}^T \mathrm{KL}\bigl(q(x_{i-1} \mid x_i, x_0) \,\|\, p_\theta(x_{i-1} \mid x_i)\bigr) \\ &- \log p_\theta(x_0 \mid x_1)\Bigr], \end{aligned}\]and with equal variances on both sides of each KL term, the \(i\)-th term is \(\frac{1}{2\tilde\beta_i}\|\tilde\mu_i(x_i, x_0) - \mu_\theta(x_i, i)\|^2\), which in the parameterization (3.1) equals \(\frac{\beta_i^2}{2\tilde\beta_i\alpha_i(1 - \bar\alpha_i)}\|\varepsilon - \varepsilon_\theta(x_i, i)\|^2\) with \(\varepsilon\) the noise of Proposition 3.1.2.
Proof. By Jensen's inequality,
\[-\log p_\theta(x_0) = -\log\int p_\theta(x_{0:T})\, dx_{1:T} \;\le\; \mathbb E_q\Bigl[-\log \frac{p_\theta(x_{0:T})}{q(x_{1:T} \mid x_0)}\Bigr],\]the expectation over the forward chain. Write \(p_\theta(x_{0:T}) = p_\theta(x_T)\prod_{i \ge 1} p_\theta(x_{i-1} \mid x_i)\) and, by the Markov property and Bayes' rule, \(q(x_{1:T} \mid x_0) = q(x_T \mid x_0)\prod_{i \ge 2} q(x_{i-1} \mid x_i, x_0)\). Pairing the factors gives the stated sum of relative entropies. The relative entropy of two Gaussians with the same covariance \(\tilde\beta_i I\) is the squared distance of the means over \(2\tilde\beta_i\), and substituting \(x_0 = (x_i - \sqrt{1 - \bar\alpha_i}\,\varepsilon)/\sqrt{\bar\alpha_i}\) into \(\tilde\mu_i\) gives \(\tilde\mu_i = \frac{1}{\sqrt{\alpha_i}}(x_i - \frac{\beta_i}{\sqrt{1 - \bar\alpha_i}}\varepsilon)\), the form (3.1) with the true noise; subtracting (3.1) leaves the stated weight. ∎Ho, Jain and Abbeel (2020), Sections 2 and 3, Equations 5 to 12. The decomposition of the bound into per-step relative entropies is from Sohl-Dickstein et al. (2015), Section 2.3, and the Jensen step is the standard variational bound of D. P. Kingma and M. Welling, "Auto-encoding variational Bayes", ICLR 2014.
Ho et al. simplify the loss by dropping the weight. They train on \(L_{\mathrm{simple}} = \mathbb E_{i, x_0, \varepsilon}\|\varepsilon - \varepsilon_\theta(x_i, i)\|^2\) with \(i\) uniform, and report better samples. This is the denoising regression of Section 2 expressed in noise units, or equivalently the clean-point loss weighted by the signal-to-noise ratio.
The likelihood bound adds the factor \(\beta_i^2/(2\tilde\beta_i\alpha_i(1 - \bar\alpha_i))\), which is larger at small noise and smaller at large noise. Removing it gives relatively more attention to high-noise steps, where the sample's global structure is decided. By Lemma 2.3.6, both losses have the same ideal minimizer. Their difference matters when capacity and training time are limited.
Training does not require running the whole reverse chain. We choose an index, construct its noisy input directly from a clean example, and score one prediction against the noise we added. Sampling is sequential: each reverse step needs the state produced by the preceding one. The same local regression problem thus supports a much longer computation at generation time.
The sampler follows the learned chain from noise to data.
Algorithm 3.3.2 DDPM-SAMPLE
(ancestral sampling; Ho, Jain and Abbeel 2020)
Picture: one step. The network reads x_i, estimates
the noise, the posterior mean is formed from it, and
fresh noise of variance beta~_i (or beta_i) is added.
x_i ---> eps_theta(x_i, i) ---> x0_hat
| |
'-----> mu_theta = posterior mean <'
|
+ sqrt(beta~_i) z
|
v
x_{i-1}
Input a network eps_theta; the rates beta_i; the
variance choice (beta~_i or beta_i).
Output a sample x_0.
1. x_T ~ N(0, I)
2. for i = T, T-1, ..., 1:
3. z ~ N(0, I) if i > 1 else z <- 0
4. mu <- (x_i - beta_i / sqrt(1 - abar_i) eps_theta(x_i, i))
/ sqrt(alpha_i)
5. x_{i-1} <- mu + sqrt(variance_i) z
6. return x_0
left 60 of 2,000 chains of Algorithm 3.3.2 on R2 with T steps, over the exact density of the
continuous process; one chain is followed; at the inspected step the true reverse transition
(a mixture) is drawn beside the sampler's Gaussian step
right the exact law of the sampler's output against p_0 (W_1 = 0.270 at T = 10), or the chain-rule
budget of its relative entropy, step by step
(the live figure needs JavaScript)
Example 3.3.3 (the sampler on R2). In the figure's default run with \(T = 10\) and the posterior variance, the exact law of the sampler's output, computed by propagating its density through the ten steps, is at Wasserstein-1 distance \(0.270\) from R2 (\(0.293\) at \(T = 5\), \(0.219\) at \(T = 25\), \(0.104\) at \(T = 100\) and \(0.003\) at \(T = 1000\)), and its relative entropy from the true law at the last noise level is \(0.099\) nats; the histogram of the 2,000 drawn chains is visibly too wide around each bump. The chain is run with the exact noise estimate of Example 2.1.6, so there is no learned-score error. The chain rule of relative entropy separates the remaining error into its sources: the relative entropy of the sampler's whole chain from the true reverse chain is the prior term, \(\mathrm{KL}(p_{t_T} \| \mathcal N(0, 1)) = 0.226\) at \(T = 10\), where the chain starts at \(\bar\alpha_T = 0.904\) rather than at pure noise, plus one term per step, the expected relative entropy of the true reverse transition \(q(x_{i-1} \mid x_i)\), a mixture of the Gaussians of Proposition 3.2.1, from the sampler's Gaussian with the same mean.
The figure's second view draws the terms: \(0.0098\) at the first step, \(0.330\) at \(i = 2\) and \(9.38\) at \(i = 1\), the last stochastic step. The large final terms reflect the shape of the reverse transition. By the law of total variance the true transition has variance \(\tilde\beta_i + c_0^2\, \mathrm{Var}(x_0 \mid x_i)\) per coordinate, with \(c_0 = \sqrt{\bar\alpha_{i-1}}\, \beta_i/(1 - \bar\alpha_i)\) the weight of \(x_0\) in the posterior mean, and the second term does not vanish as \(T\) grows at the steps where the posterior of \(x_0\) is still undecided: at \(T = 1000\) the last three terms are \(0.0375\), \(0.0708\) and \(0.2038\).
The other variance, \(\beta_i\), is larger than \(\tilde\beta_i\) by the factor \((1 - \bar\alpha_i)/(1 - \bar\alpha_{i-1})\) and is the better choice at the last steps for a law with a density: with it the terms at \(T = 10\) sum to \(0.047\) instead of \(9.92\), the exact output law is at distance \(0.267\), and at \(T = 1000\) the sum is \(0.0068\) against \(0.443\). Ho et al. noted that \(\tilde\beta_i\) is optimal for a point mass and \(\beta_i\) for \(\mathcal N(0, I)\); the ledger shows the two cases step by step.
This sum bounds the exact relative entropy of the output by data processing, \(0.273\) against \(0.108\) with \(\beta_i\) at \(T = 10\), and the slack is the last steps' variance term, which the output law does not see. Section 5 is about doing better than this with the same number of network evaluations. ◁
The program below recomputes the numbers of Examples 3.1.3 and 3.2.2.
# One step of the DDPM chain on R2, with T = 10, at the first step from
# pure noise (i = 9 in 0-based indexing), for the particle at x_t = -1.1374.
#
# beta_i linear from 1e-4 to 0.02; abar_i = prod_{j <= i} (1 - beta_j)
# x_t = sqrt(abar_i) x_0 + sqrt(1 - abar_i) eps
# eps_hat = -sigma score, x0_hat = (x_t - sigma eps_hat) / alpha
# mu~ = (x_t - beta_i eps_hat / sigma) / sqrt(1 - beta_i)
# beta~_i = (1 - abar_{i-1}) / (1 - abar_i) beta_i
#
# Picture: the chain of T = 10 steps, with the step inspected.
#
# x_0 <- x_1 <- ... <- x_8 <- x_9 <- x_10 ~ N(0, I)
# ^^^^^
# inspected: i = 9
import math
W, M, S = (0.45, 0.35, 0.20), (-1.4, 0.2, 1.6), (0.25, 0.15, 0.30)
def chain(T, b1=1e-4, bT=0.02):
beta = [b1 + (bT - b1) * i / (T - 1) for i in range(T)]
abar, acc = [], 1.0
for b in beta:
acc *= 1 - b
abar.append(acc)
return beta, abar
def denoise(x, a, s, comps): # E[x_0 | x_t = x] for a mixture of Gaussians
logw, means, gains = [], [], []
for w, m, sd in comps:
v = a * a * sd * sd + s * s
logw.append(math.log(w) - 0.5 * (x - a * m) ** 2 / v - 0.5 * math.log(2 * math.pi * v))
means.append(m + (a * sd * sd / v) * (x - a * m))
gains.append(v)
top = max(logw)
r = [math.exp(l - top) for l in logw]
z = sum(r)
return sum(ri / z * mi for ri, mi in zip(r, means))
beta, abar = chain(1000)
print(f"T = 1000: abar_T = {abar[-1]:.3e}, sqrt(abar_T) = {math.sqrt(abar[-1]):.4f}, abar_500 = {abar[499]:.4f}")
beta, abar = chain(10)
i, x = 9, -1.1374
a, s = math.sqrt(abar[i]), math.sqrt(1 - abar[i])
x0_hat = denoise(x, a, s, list(zip(W, M, S)))
eps_hat = (x - a * x0_hat) / s
mu = (x - beta[i] / s * eps_hat) / math.sqrt(1 - beta[i])
beta_tilde = (1 - abar[i - 1]) / (1 - abar[i]) * beta[i]
print(f"T = 10, step i = {i}: abar_i = {abar[i]:.4f}, alpha = {a:.4f}, sigma = {s:.4f}, beta_i = {beta[i]:.4f}")
print(f" x_t = {x}: x0_hat = {x0_hat:.3f}, eps_hat = {eps_hat:.3f}, mu~ = {mu:.3f}, sd beta~ = {math.sqrt(beta_tilde):.3f}, sd beta = {math.sqrt(beta[i]):.3f}")
T = 1000: abar_T = 4.036e-05, sqrt(abar_T) = 0.0064, abar_500 = 0.0786
T = 10, step i = 9: abar_i = 0.9037, alpha = 0.9507, sigma = 0.3103, beta_i = 0.0200
x_t = -1.1374: x0_hat = -1.324, eps_hat = 0.391, mu~ = -1.174, sd beta~ = 0.127, sd beta = 0.141
Example 3.3.4 (the sampler on the ε-model). We follow the \(\varepsilon\)-model of Section 11, widths (64, 64, 128) and 2,524,673 parameters, trained for forty epochs on FashionMNIST at \(28 \times 28\) on DDPM's chain of 1,000 steps, \(\beta_i\) linear from \(10^{-4}\) to \(0.02\). Algorithm 3.3.2, with the variance \(\beta_{i-1}\) of Example 3.2.2, runs for the full 1,000 steps from six fixed draws of \(\mathcal N(0, I)\), and every fifth state is kept with its noise estimate and the clean estimate of Tweedie's formula.
This chain reaches nearly pure noise. At the first step, the model is asked about an input of root-mean-square size \(0.986\), as its training inputs at that index were, and its noise estimate has root-mean-square \(0.990\). Tweedie's \(\hat x_0 = (x_T - \sigma_T\hat\varepsilon)/\alpha_T\) divides by \(\alpha_T = 0.0064\), and the first clean estimate is amplified noise, at root-mean-square distance \(3.13\) per pixel from the final image.
The clean estimates \(\hat x_0\) are blurred class averages a quarter of the way down the chain, become recognizable garments by its middle, and stop changing by more than \(0.05\) per pixel from step \(974\) of \(1{,}000\) on, at index \(25\), after which the steps mostly refine pixels within an established shape. The state itself barely changes size along the way, from \(0.986\) at the start to \(0.801\) at the image, and the five steps between kept states move it by \(0.316\) per pixel at the start. Section 5.4 measures the same model at every number of steps. ◁
six chains of Algorithm 3.3.2 on the epsilon-model trained on DDPM's chain of 1,000 steps, from
N(0, I) at index 999 to the image at index 0, every fifth state kept, shown as the state x_t, the
clean estimate x_0-hat of Tweedie's formula, or the noise estimate, at any kept step, with the
change map x_{k+1} - x_k of what the five steps added under each; the strip gives the size of the
state, of the correction and of the five-step change along the chain; the clean estimates stop
changing by more than 0.05 per pixel from step 974 of 1,000 on
(the live figure needs JavaScript and loads 2.08 MB of data)
Example 3.3.5 (the loss by noise level, in practice). We train the \(\varepsilon\)-model of Section 11, widths (64, 64, 128), for forty epochs on \(L_{\mathrm{simple}}\) with \(i\) uniform, on FashionMNIST at \(28 \times 28\) and the 150-step chain, with a checkpoint every thousand steps. We measure the error of its noise estimate, \(\mathbb E\|\varepsilon - \varepsilon_\theta\|^2\) per pixel, on 2,000 held-out images with fixed noise draws, in fifteen bins of the chain's index.
The figure below shows what the model reached after forty epochs, in fifteen bins of its chain: \(0.339\) at the clean end, \(0.0542\) in the middle and \(0.0223\) at the noise end, against two exact references. The exact posterior mean over the training set, the minimizer of the same objective (Section 2.5), scores \(124.2\), \(0.1545\) and \(0.0252\) on the same held-out images: it is a worse noise predictor than the network from bin \(15\) down, that is, in every bin, narrowly at the noise end, where its posterior is no longer owned by one training image, and by a factor of hundreds at the clean end, where it returns the nearest training image, a fixed distance away, and the noise estimate divides that distance by \(\sigma\).
The Gaussian law with the data's covariance has floor \(0.835\) and \(0.0349\) at the two ends, which is also the error of the best linear denoiser: the network is far below it at low noise, which is the non-Gaussian structure it has learned, and nearer to it at the noise end, where every law looks more nearly Gaussian.
The right panel draws the weight of Theorem 3.3.1 that \(L_{\mathrm{simple}}\) drops: on the 150-step chain it is \(1.168\) at the first step and \(0.0131\) at the last, and with the model's profile the bound's diffusion term is \(1084.4\) nats per image against the prior term of Example 3.1.5, \(70.05\) over the held-out images (\(69.64\) from the training set's moments, which this figure prints); on DDPM's chain of 1,000 steps the same architecture reaches \(0.00041\) at the noise end and the prior term is \(0.011\). ◁
left the held-out loss of the epsilon-model by noise level, in fifteen bins of its chain, at every
checkpoint of training, against the exact posterior mean over the training set (the
memorizing minimizer) and the Gaussian floor from the data's covariance
right the per-step weight of the evidence lower bound of Theorem 3.3.1 against the unit weight of
L_simple, or the training curve
(the live figure needs JavaScript and loads 0.13 MB of data)
The bound in continuous time
What happens to the likelihood bound as the time grid becomes finer? Kingma, Salimans, Poole and Ho take \(T \to \infty\) in Theorem 3.3.1. The sum becomes an integral, and a useful simplification appears: the bound depends on the schedule only through its endpoints. Changing the clock changes how we estimate the integral, rather than the integral itself.
Theorem 3.4.1 (the continuous-time bound). Let the forward process be the Gaussian kernel of Section 1 on a schedule with log signal-to-noise ratio \(\lambda_t\) decreasing from \(\lambda_0\) to \(\lambda_1\), and let the reverse model be defined through a denoiser \(D_\theta(x, \lambda)\). Then, with \(x_\lambda\) the noised point at the time where the log signal-to-noise ratio is \(\lambda\),
\[\begin{aligned} -\log p_\theta(x_0) \;\le\; &\mathrm{KL}\bigl(p_1(\cdot \mid x_0) \,\|\, \mathcal N(0, I)\bigr) + \mathbb E\bigl[-\log p_\theta(x_0 \mid x_{\lambda_0})\bigr] \\ &+ \tfrac12 \int_{\lambda_1}^{\lambda_0} \mathbb E_{\varepsilon}\bigl\|\varepsilon - \varepsilon_\theta(x_\lambda, \lambda)\bigr\|^2\, d\lambda . \end{aligned}\]The integral term, the diffusion loss, is invariant under reparameterizations of time: two schedules with the same endpoints \(\lambda_0, \lambda_1\) give the same bound. In the units of \(x_0\) it reads \(\tfrac12 \int e^{\lambda}\, \mathbb E\|x_0 - D_\theta\|^2\, d\lambda\), an integral against the signal-to-noise ratio.
Proof sketch. Take the sum of Theorem 3.3.1 on a grid of \(T\) times, write the weight of the \(i\)-th term in the \(x_0\) units of Proposition 2.4.2, and observe that it equals \(\tfrac12(\mathrm{SNR}(t_{i-1}) - \mathrm{SNR}(t_i)) = \tfrac12\lvert\Delta\, e^{\lambda}\rvert\) to first order, a telescoping increment of the signal-to-noise ratio. Summing against \(\|x_0 - D_\theta\|^2\) and letting the grid refine gives a Riemann–Stieltjes integral in \(\mathrm{SNR}\), which the substitution \(d\,\mathrm{SNR} = e^\lambda d\lambda\) turns into the stated integral in \(\lambda\) after absorbing \(e^\lambda\) into the \(\varepsilon\) units: \(\|x_0 - D_\theta\|^2 e^\lambda = \|\varepsilon - \varepsilon_\theta\|^2\). The invariance is then the statement that the integrand depends on \(t\) only through \(\lambda\). ∎Kingma, Salimans, Poole and Ho, "Variational diffusion models" (2021), Section 3 and Appendix: "the variational lower bound (VLB) simplifies to a remarkably short expression in terms of the signal-to-noise ratio of the diffused data" and "the continuous-time VLB is invariant to the noise schedule, except for the signal-to-noise ratio at its endpoints". Their Table 1 reports \(2.65\) bits per dimension on the CIFAR-10 test set, \(2.49\) with extensive data augmentation, and \(3.72\) and \(3.40\) on ImageNet at \(32 \times 32\) and \(64 \times 64\). The earlier likelihood of Song et al. (2021) was \(2.99\) bits per dimension, with a sample FID of \(2.20\) on CIFAR-10; the maximum-likelihood weighting of Y. Song, C. Durkan, I. Murray and S. Ermon, "Maximum likelihood training of score-based diffusion models", NeurIPS 2021 (arXiv 2101.09258), reaches \(2.83\) without augmentation.
The theorem explains the second figure of Section 3.1: in likelihood terms a schedule only chooses how training time is distributed over \(\lambda\) (the figure's second view), and the endpoints decide the two boundary terms. It also places \(L_{\mathrm{simple}}\): by Kingma and Gao's result quoted in Section 2.4, a denoising loss with a weight that is monotone in \(\lambda\) is the bound of Theorem 3.4.1 for a noise-augmented data law, so DDPM's unweighted \(\varepsilon\)-loss, whose weight in the \(x_0\) units is \(e^\lambda\), is a likelihood objective for data that have been slightly blurred. For a law with a density the integrand tends to \(d\) as \(\lambda \to +\infty\) (the floor in the units of the noise tends to one per coordinate, Proposition 2.3.5 with \(I(p_t) \to I(p_0)\) finite; the figure below reads \(0.998\) at \(\lambda = 9.21\)), which is why the clean endpoint \(\lambda_0\) must be finite and the reconstruction term closes the bound there; at the noise end the integrand vanishes like \(e^{\lambda}\, \mathrm{Var}(x_0)\), so a finite \(\lambda_1\) costs only the prior term.
left the running diffusion loss L(t) of R2 with the exact noise estimate against t, for the
linear-beta schedule and the cosine schedule, from the time each enters the range of DDPM's
chain: two shapes, 4.068 against 3.701 nats at t = 0.5, meeting at 4.12 nats at t = 1
right against lambda, the one integrand both schedules integrate, half of R2's floor E|eps − eps_hat|^2
(0.998 at lambda = 9.21, 0.00006 at −10.12), with the staircase of the sum of Theorem 3.3.1 on
DDPM's chain re-spaced to K steps: 4.57 nats at K = 1000, 13.2 at 100, 79.7 at 25 and 454 at
10, against the integral 4.12
(the live figure needs JavaScript)
DDPM as score matching
We derived DDPM through a latent-variable model and a likelihood bound. We can now read the same construction in the language of Section 2: its noise predictor is a score estimator, and its sampling step approximates the reverse SDE. The two descriptions meet in the following proposition.
Proposition 3.5.1 (the two objectives coincide). \(L_{\mathrm{simple}}\) at step \(i\) is the denoising score-matching objective (2.3) at the time \(t_i\), in the units \(\varepsilon_\theta = -\sigma_{t_i} s_\theta\), and the mean (3.1) of the learned reverse step is, to first order in \(\beta_i\), one Euler–Maruyama step of Anderson's reverse SDE of Section 4.2: \(\mu_\theta(x_i, i) = x_i + \tfrac12\beta_i\, x_i + \beta_i\, s_\theta(x_i, t_i) + O(\beta_i^2)\).
Proof. The first statement is Theorem 2.3.4 with \(x_i = \alpha_{t_i}x_0 + \sigma_{t_i}\varepsilon\) and \(s_\theta = -\varepsilon_\theta/\sigma_{t_i}\). For the second, \(1/\sqrt{\alpha_i} = 1 + \tfrac12\beta_i + O(\beta_i^2)\) and \(\beta_i\varepsilon_\theta/\sqrt{1 - \bar\alpha_i} = -\beta_i s_\theta\); the reverse SDE of Theorem 4.2.1 with \(f = -\tfrac12\beta\) and \(g^2 = \beta\), run backward over a step \(\Delta t\) with \(\beta\Delta t = \beta_i\), has drift \(-\tfrac12\beta_i x - \beta_i s\) in forward time, hence \(+\tfrac12\beta_i x + \beta_i s\) in the reverse step. ∎
DDPM therefore has three compatible descriptions: a latent-variable model trained through a variational bound, a score model trained by denoising regression, and a discrete approximation to the reverse SDE. Dieleman's essay develops these perspectives. Section 4.3 adds a fourth by constructing a deterministic flow from the same learned score.
Remark 3.5.2 (confusion: why the steps must be small). The learned reverse step is Gaussian. The true reverse transition \(q(x_{i-1} \mid x_i)\), averaged over the unknown \(x_0\), is a mixture of the Gaussians of Proposition 3.2.1 and is not Gaussian; it is close to Gaussian only when \(\beta_i\) is small, because then the posterior of \(x_0\) barely changes the shape of the step.
Nakkiran et al. quantify it: the relative entropy of the true reverse transition from its Gaussian approximation is \(O(\sigma^4)\) per step of noise standard deviation \(\sigma\), so that the total error over a schedule of \(T\) steps of variance \(\sigma^2 = 1/T\) each is \(O(1/T)\), and the number of steps needed for a fixed accuracy grows as \(\Omega(1/\sigma^2)\).P. Nakkiran, A. Bradley, H. Zhou and M. Advani, "Step-by-step diffusion: an elementary tutorial", arXiv 2406.08929 (2024), Claim 1 and Lemma 1 in Section 2.1: for small steps, \(p(x_{t-\Delta t} \mid x_t = z) \approx \mathcal N(z + \sigma_q^2\Delta t\, \nabla\log p_t(z), \sigma_q^2\Delta t)\), with a KL error of order \(\sigma^4\); "the number of steps only grows as \(\Omega(1/\sigma^2)\)". The first figure of Section 3.3 prints the per-step terms: their \(O(\sigma^4)\) decay is visible from \(i = T - 1\) down to a few steps from the end, and what does not decay is the last steps' variance mismatch, a different error.
The Gaussian approximation is also why the reverse step's mean must contain the score and nothing else: the posterior mean of a small Gaussian step is the point moved by the score, which is Tweedie's formula once more, applied to a single step rather than to the whole chain. Section 4 removes the approximation by passing to the limit, and Section 5 shows that a deterministic step tolerates much larger \(\beta_i\) than a stochastic one, which is the reason it needs so many fewer steps.