8. Few steps
Distillation, consistency and flow maps Draft
The flow map and the average velocity
A sampler that follows the local velocity needs repeated network evaluations because that velocity changes along the path. To cross a large interval in one step, we can instead learn where the path ends. The resulting function is a flow map: it sends a point at one time to its position at another.
This viewpoint connects many few-step methods developed from 2022 to 2026. Some learn the endpoint directly; others learn the average velocity needed to reach it. We will first establish the identities behind these targets, then examine how the methods construct training signals for them.
Keep the flow map separate from the denoiser. The denoiser estimates which clean point could have produced an observation under the corruption process. The flow map returns the particular endpoint assigned to that observation by a deterministic trajectory. At pure noise, the denoiser can return the same data mean everywhere, while the flow map must send different noise points to different samples. A one-step generator needs the latter kind of target.
left the denoiser D(x, t) (ink) and the flow map X_{t→0}(x) (green) of R2 against x_t, at t = 1: D is flat
at −0.24, the mean of R2, while X is the quantile map from N(0, 1) to R2, sending −1 to −1.49, 0 to
0.04 and 1 to 1.35; the identity and the bump means as hairlines, p_t as a wash along the bottom
right the laws of the outputs over x_t ~ p_t: the flow map's law is p₀ at every t (W₁ 0.002 on the grid,
the discretization); the denoiser's law at t = 1 is the point mass at −0.24, at W₁ 1.044 from p₀,
and has sd 0.338, 0.923 and 1.170 at t = 0.8, 0.5 and 0.2 against 1.185 for R2
(the live figure needs JavaScript)
Definition 8.1.1 (flow map). For the probability-flow ODE \(\dot x = u_t(x)\) of Section 4.3, the flow map \(X_{s \to t}: \mathbb R^d \to \mathbb R^d\) sends the point of a trajectory at time \(s\) to its point at time \(t\): \(X_{s \to t}(x(s)) = x(t)\) for every solution. It satisfies \(X_{s \to s} = \mathrm{id}\), the semigroup identity \(X_{t \to r} \circ X_{s \to t} = X_{s \to r}\), and \(X_{t \to s} = X_{s \to t}^{-1}\).
The two equations describe two ways to inspect the same trip. Hold the departure point fixed and extend the arrival time: the endpoint moves with the velocity there. Alternatively, move the departure point a short way along its trajectory while keeping the arrival time fixed: the final destination stays the same. The Lagrangian and Eulerian equations express these two observations.
Proposition 8.1.2 (the Lagrangian and Eulerian equations). The flow map is characterized by either of two equations: the Lagrangian one, in its second argument,
\[\partial_t X_{s \to t}(x) = u_t\bigl(X_{s \to t}(x)\bigr), \qquad X_{s \to s}(x) = x,\]and the Eulerian one, in its first argument,
\[\partial_s X_{s \to t}(x) + \nabla X_{s \to t}(x)\, u_s(x) = 0, \qquad X_{t \to t}(x) = x .\]Proof. The Lagrangian equation is the ODE itself. For the Eulerian one, differentiate the semigroup identity \(X_{s \to t}(X_{r \to s}(y)) = X_{r \to t}(y)\) in \(s\) at a fixed \(y\): the right side does not depend on \(s\), so \(0 = \partial_s X_{s \to t}(x) + \nabla X_{s \to t}(x)\, \partial_s X_{r \to s}(y)\) with \(x = X_{r \to s}(y)\), and \(\partial_s X_{r \to s}(y) = u_s(x)\) by the Lagrangian equation. Each equation with its boundary condition determines the map, by uniqueness for the ODE in the first case and for the linear transport equation in the second. ∎N. M. Boffi, M. S. Albergo and E. Vanden-Eijnden, "Flow map matching with stochastic interpolants: a mathematical framework for consistency models", Transactions on Machine Learning Research (arXiv 2406.07507, first version June 2024, titled "Flow map matching"), Definition 3.3 and the Lagrangian and Eulerian characterizations; the corresponding objectives are their Lagrangian map distillation and Eulerian map distillation, and the semigroup identity gives a third, progressive one.
Definition 8.1.3 (average velocity). For \(r \ne t\), the average velocity from time \(t\) to time \(r\) at the point \(x\) is
\[u(x, r, t) \;=\; \frac{X_{t \to r}(x) - x}{r - t} \;=\; \frac{1}{r - t}\int_t^r u_\tau\bigl(X_{t \to \tau}(x)\bigr)\, d\tau,\]extended by \(u(x, t, t) = u_t(x)\). One step of length \(r - t\) with the average velocity lands exactly on the flow map: \(x + (r - t)\, u(x, r, t) = X_{t \to r}(x)\).
The instantaneous velocity is a tangent to the path; the average velocity describes the chord across the whole interval. They agree on a straight, constant-speed trajectory and can differ greatly on a curved one. The average also depends on how far we intend to travel, which is why a network learning it needs both times. The identity below relates the chord to the tangent without requiring us to integrate the full path for every target.
left the (t, x) plane of R2 on the linear path, the noise at the left and the data at the right: forty flow
lines, and the trajectory through (z, t) = (0.9, 0.8) with its tangent of slope v = 0.927 (dotted,
landing at 0.344 at r = 0.2) against its chord to r of slope u = −0.446 (green, landing at 1.167,
where the flow lands); or the trip from x = 0.6 at s = 1, which lands at 0.320, with its departure
moved to s = 0.7 (∂_s X = −0.273 and ∇X · u_s = +0.273 cancel) and the trip from the same x at
s = 0.7 landing at 0.953; or the same trip composed at m = 0.5 (half slopes 0.475 and 0.085, their
time-weighted mean 0.280) against the jump whose second half is taken from the original x, which
lands at 0.884
right the chord's slope u(z, r, t) against r (0.927 at r = t, 0 near r = 0.37, −0.740 at r = 0), or the
landing against s or m, the trip's arrival as a flat green line
(the live figure needs JavaScript)
Theorem 8.1.4 (the MeanFlow identity). Along a trajectory \(x_t\) of the ODE, for fixed \(r\),
\[u(x_t, r, t) \;=\; u_t(x_t) - (t - r)\, \frac{d}{dt}\, u(x_t, r, t), \qquad \frac{d}{dt} = \partial_t + u_t(x_t) \cdot \nabla_x ,\]the derivative being the total derivative along the trajectory.
Proof. By Definition 8.1.3, \((t - r)\, u(x_t, r, t) = \int_r^t u_\tau(x_\tau)\, d\tau\), because \(X_{t \to \tau}(x_t) = x_\tau\) on the trajectory. Differentiate both sides in \(t\): the left gives \(u + (t - r)\, \frac{d}{dt}u\) and the right gives \(u_t(x_t)\). ∎Z. Geng, M. Deng, X. Bai, J. Z. Kolter and K. He, "Mean flows for one-step generative modeling", NeurIPS 2025 (arXiv 2505.13447), Equation 6, "the MeanFlow Identity", \(u(z_t, r, t) = v(z_t, t) - (t - r)\frac{d}{dt}u(z_t, r, t)\), derived from \((t - r)\, u(z_t, r, t) = \int_r^t v(z_\tau, \tau)\, d\tau\); their \(v\) is the instantaneous velocity, in the same convention as this series, \(z_t = (1 - t) x + t \varepsilon\) with \(t = 0\) at the data.
The average velocity is defined by an integral along a trajectory we do not know. The MeanFlow identity replaces that integral with a local relation involving the instantaneous velocity and a derivative of \(u\). The velocity can be learned from the conditional targets of Section 6, and the derivative can be computed as a Jacobian–vector product along \((u_t, 1)\). This makes the identity useful as a training condition.
Consistency asks all points on one trajectory to name the same clean endpoint. Agreement alone would be too weak: a function returning one constant for every input would also agree everywhere. The boundary condition anchors each clean point to itself. Together, agreement along trajectories and the boundary values identify the desired map.
Proposition 8.1.5 (the consistency function). The map \(f(x, t) = X_{t \to 0}(x)\) sends every point of a trajectory to the trajectory's origin, \(f(x_t, t) = x_0\), and it is the unique solution of
\[\partial_t f(x, t) + \nabla f(x, t)\, u_t(x) = 0, \qquad f(x, 0) = x,\]which is the Eulerian equation of Proposition 8.1.2 with the target time fixed at \(0\). Equivalently, \(\frac{d}{dt} f(x_t, t) = 0\) along every trajectory.
Proof. The first statement is Definition 8.1.1; the equation is the Eulerian one with \(t = 0\) as the target, and the total derivative form is the chain rule. ∎
left 40 flow lines of R3 on the linear path from noise to data (200 more faint); N Euler or Heun
steps with the instantaneous velocity (dashed, hollow endpoints), or N jumps with the exact
average velocity (green: the jumps of the flow map are the accent), or the consistency
function's arrows (green) sending points of one line to its origin; a probe on one line, its
exact flow line blue, with its chord to time r (green)
right the endpoint error of Euler and Heun against the number of steps with reference slopes; or
the instantaneous and the average velocity as two fields; or u and v along the probe's line
(the live figure needs JavaScript)
Example 8.1.6 (one step, three ways). In the figure's default view a single Euler step from \(t = 1\) with the instantaneous velocity lands at a root-mean-square distance of \(0.953\) from the flow's endpoints, which is of the order of the distance between the moons: the velocity at the noise end points toward the centre of mass of the data, not toward the moon the flow line will choose. Eight Euler steps still miss by \(0.120\) and eight Heun steps by \(0.036\). One jump with the exact average velocity \(u(z, 0, 1) = (X_{1 \to 0}(z) - z)/(0 - 1) = z - X_{1 \to 0}(z)\), which lands at \(z + (0 - 1)\, u(z, 0, 1) = X_{1 \to 0}(z)\), is exact, as Definition 8.1.3 says it must be, and so is any sequence of jumps with the average velocities over the sub-intervals.
The consistency view shows the same object from the middle of the line: every point of a flow line is sent to the same origin. At the probe of the default view, \(z = (0.90, 0.60)\) at \(t = 0.8\) with \(r = 0.2\), the instantaneous velocity is \((0.903, 0.757)\) and the average velocity to \(r\) is \((-0.044, 0.786)\), with opposite first coordinates; the identity of Theorem 8.1.4 holds there to \(7.1 \times 10^{-9}\), and one step with \(v\) lands at \((0.358, 0.146)\) where the flow lands at \((0.927, 0.128)\).
The view 'the two fields' draws the two regression targets over the plane, both in the sampler's direction of motion: the instantaneous velocity at \(t = 1\), which is \(D(x) - x\) with \(D\) equal to the mean of the data there and so points toward the data's centre of mass from everywhere, and the one-step jump \(X_{1 \to 0}(x) - x\), which points toward the basin each start reaches; their mean angle, over the cells where both exceed \(0.05\) in length, is \(36.0^\circ\). The fitted orders of the error curve are \(1.04\) and \(1.87\). The program below checks the identity of Theorem 8.1.4 numerically on R2, where the trajectory and the average velocity are computed by a fine Runge–Kutta integration and the total derivative by finite differences along the trajectory. ◁
# The MeanFlow identity on R2 under the linear path, checked numerically.
#
# average velocity u(z, r, t) = (1/(t - r)) int_r^t v(x_tau, tau) dtau,
# x_tau the probability-flow trajectory through (z, t)
# identity u(z, r, t) = v(z, t) - (t - r) d/dt u(z_t, r, t),
# d/dt the total derivative along the trajectory.
#
# Picture: the trajectory through (z, t), and the chord from
# (z, t) to its point at time r, whose slope is u(z, r, t).
#
# x | . z at t
# | .' \ chord: average velocity
# | .' \
# | .' o x_r at r
# +----------------- time
# r t
import math
W, M, S = (0.45, 0.35, 0.20), (-1.4, 0.2, 1.6), (0.25, 0.15, 0.30)
def velocity(x, t): # u_t = -D + eps_hat for alpha = 1 - t, sigma = t
a, s = 1 - t, t
logw, means, scores = [], [], []
for w, m, sd in zip(W, M, S):
v = a * a * sd * sd + s * s
logw.append(math.log(w) - 0.5 * (x - a * m) ** 2 / v - 0.5 * math.log(2 * math.pi * v))
means.append(m + (a * sd * sd / v) * (x - a * m))
scores.append((a * m - x) / v)
top = max(logw)
r = [math.exp(l - top) for l in logw]
z = sum(r)
D = sum(ri / z * mi for ri, mi in zip(r, means))
score = sum(ri / z * si for ri, si in zip(r, scores))
return -D - s * score
def flow(x, t_from, t_to, n=400): # RK4 for the probability-flow ODE
h = (t_to - t_from) / n
for i in range(n):
t = t_from + i * h
k1 = velocity(x, t); k2 = velocity(x + h / 2 * k1, t + h / 2)
k3 = velocity(x + h / 2 * k2, t + h / 2); k4 = velocity(x + h * k3, t + h)
x += h / 6 * (k1 + 2 * k2 + 2 * k3 + k4)
return x
def average(z, r, t): return (flow(z, t, r) - z) / (r - t)
z, r, t, h = 0.9, 0.2, 0.8, 1e-4
v = velocity(z, t)
# the total derivative of u(z_t, r, t) in t: move z along the trajectory by v h
du = (average(z + v * h, r, t + h) - average(z - v * h, r, t - h)) / (2 * h)
print(f"z = {z}, r = {r}, t = {t}")
print(f" instantaneous velocity v(z, t) = {v:.5f}")
print(f" average velocity u(z, r, t) = {average(z, r, t):.5f}")
print(f" v - (t - r) du/dt = {v - (t - r) * du:.5f}")
print(f" one step with v lands at {z + (r - t) * v:.4f}; with u at {z + (r - t) * average(z, r, t):.4f}; the flow at {flow(z, t, r):.4f}")
z = 0.9, r = 0.2, t = 0.8
instantaneous velocity v(z, t) = 0.92728
average velocity u(z, r, t) = -0.44555
v - (t - r) du/dt = -0.44555
one step with v lands at 0.3436; with u at 1.1673; the flow at 1.1673
The two velocities at \((z, t) = (0.9, 0.8)\) differ in sign: the instantaneous velocity of the flow there points toward the centre, while the chord of the trajectory over \([0.2, 0.8]\) points outward, because the trajectory first drifts inward and then bends toward the bump of R2 at \(1.6\). A one-step sampler must know the second number and not the first.
Algorithm 8.1.7 MEANFLOW-TRAIN
(Geng, Deng, Bai, Kolter and He 2025; from scratch)
Picture: the network reads (x_t, r, t); its target is
the conditional velocity corrected by its own total
derivative, taken as a Jacobian-vector product and
held fixed (stop-gradient) when the loss is formed.
(x_0, z) --> x_t = (1-t) x_0 + t z, v = z - x_0
|
v
u_theta(x_t, r, t) ------------------> loss
| ^
JVP along (v, 0, 1): dudt |
| |
'--> target = v - (t-r) dudt (no gradient)
Input samples of p_0; a law for the pair (r, t) with
r <= t, a fraction of pairs with r = t; a
network u_theta(x, r, t).
Output theta with u_theta ~ the average velocity.
1. repeat until converged:
2. draw x_0, z ~ N(0, I), (r, t)
3. x_t <- (1 - t) x_0 + t z; v <- z - x_0
4. dudt <- JVP of u_theta at (x_t, r, t) along (v, 0, 1)
5. target <- stopgrad( v - (t - r) dudt )
6. take a gradient step on |u_theta(x_t, r, t) - target|^2
Line 5 uses Theorem 8.1.4 with the conditional velocity \(z - x_0\) in place of the marginal velocity. As in Algorithm 6.2.4, least squares averages over the pairs consistent with \((x_t, t)\), recovering \(u_t\). Pairs with \(r = t\) train the network's diagonal to match the ordinary flow-matching velocity.
The stop-gradient holds the target fixed during each update, so that update takes the form of a regression. Across updates, however, the target changes with the model. This is a source of both the method's usefulness and its training difficulties; Section 8.4 returns to it.
Distillation
Distillation begins with a trained diffusion model, the teacher, and trains a student to cover more of its path in each step. The early methods reduce the teacher's step count this way. Related methods can also build their own targets without a teacher. Their training conditions draw on the flow map's equations and composition rule from Section 8.1.
A teacher supplies coherent pairings between starts and endpoints. The student is asked to reproduce the endpoint of a particular trajectory, which can remain diverse even under squared-error training. It also inherits an approximation: the teacher's learned field and numerical solver determine the targets. Matching them accurately establishes fidelity to that teacher, while fidelity to the data is a further question.
Progressive distillation. Salimans and Ho train a student to reproduce two of its teacher's DDIM steps in one. That student becomes the next teacher, halving the step count each round, from 8192 steps down to 4 or 1. The target is the composition \(X_{t \to t + 2h} = X_{t + h \to t + 2h} \circ X_{t \to t + h}\), with the teacher evaluating the right side. Their \(v\)-parameterization, introduced in Proposition 2.4.1, helps keep these large steps stable. On CIFAR-10 the method reaches an FID of \(9.12\) in one step, \(4.51\) in two, \(3.00\) in four and \(2.57\) in eight.T. Salimans and J. Ho, "Progressive distillation for fast sampling of diffusion models", ICLR 2022 (arXiv 2202.00512), Algorithm 2 with the line "\(N \leftarrow N/2\)", and Table 2.
Consistency models. Song, Dhariwal, Chen and Sutskever learn the consistency function of Proposition 8.1.5 directly. They use a network \(f_\theta(x, t)\) with the boundary condition \(f_\theta(x, 0) = x\) built in by a skip connection. Training asks that its predictions at two adjacent points on one trajectory agree: \(f_\theta(x_{t_{n+1}}, t_{n+1}) \approx f_{\theta^-}(x_{t_n}, t_n)\), where \(x_{t_n}\) is obtained from \(x_{t_{n+1}}\) by one step of the teacher's ODE solver (consistency distillation) or, without a teacher, by using the same pair \((x_0, z)\) at both times (consistency training), and \(\theta^-\) is a slowly moving copy of the weights.
Sampling takes one evaluation, \(f_\theta(z, 1)\), or a few, by re-noising the sample to an intermediate time and applying \(f_\theta\) again. The condition imposed is the Eulerian equation in discrete form.
On CIFAR-10 distillation reached \(3.55\) in one step and \(2.93\) in two, training \(8.70\) and \(5.83\); the improved training recipe of Song and Dhariwal, without any teacher, reached \(2.51\) and \(2.24\) on CIFAR-10 and \(3.25\) and \(2.77\) on ImageNet at \(64 \times 64\).Y. Song, P. Dhariwal, M. Chen and I. Sutskever, "Consistency models", ICML 2023 (arXiv 2303.01469), Table 1; Y. Song and P. Dhariwal, "Improved techniques for training consistency models", ICLR 2024 (arXiv 2310.14189): "FID scores of 2.51 and 3.25 on CIFAR-10 and ImageNet 64×64 respectively in a single sampling step … Through two-step sampling, we further reduce FID scores to 2.24 and 2.77".
Continuous-time consistency. As the two adjacent times merge, the discrete condition becomes the differential one, \(\frac{d}{dt}f_\theta(x_t, t) = 0\). Training uses a Jacobian–vector product along the trajectory's tangent, a signal that proved unstable at scale.
Lu and Song stabilized it with a reparameterization of the path, TrigFlow, \(x_t = \cos t\, x_0 + \sin t\, z\) on \(t \in [0, \pi/2]\) with \(z \sim \mathcal N(0, \sigma_d^2 I)\), in which the EDM preconditioning of Remark 2.4.3 becomes \(c_{\mathrm{skip}} = \cos t\), \(c_{\mathrm{out}} = -\sigma_d\sin t\), \(c_{\mathrm{in}} = 1/\sigma_d\), together with a normalization of the tangent, an adaptive weighting and a warm-up of the derivative term.
The resulting method trains at 1.5 billion parameters on ImageNet at \(512 \times 512\) and reaches, with two sampling steps, FIDs of \(2.06\) on CIFAR-10, \(1.48\) on ImageNet \(64 \times 64\) and \(1.88\) on ImageNet \(512 \times 512\), "narrowing the gap in FID scores with the best existing diffusion models to within 10%".C. Lu and Y. Song, "Simplifying, stabilizing and scaling continuous-time consistency models", ICLR 2025 (arXiv 2410.11081), Section 3 for TrigFlow and Section 6 for the numbers; the consistency model is \(f_\theta(x_t, t) = \cos t\, x_t - \sin t\, \sigma_d F_\theta(x_t/\sigma_d, c_{\mathrm{noise}}(t))\).
Composition offers another check on a proposed large jump: one full jump should agree with two half jumps, including the change in position after the first half. This is stronger than asking two predictions at the original input to agree. The small-step limit supplies the local velocity that anchors the construction.
Shortcut models. Frans, Hafner, Levine and Abbeel condition the network on the step size as well: \(s_\theta(x, t, d)\) is the average velocity over a step of length \(d\), trained with the flow-matching loss at \(d = 0\) and the self-consistency condition \(s(x, t, 2d) = \tfrac12\bigl(s(x, t, d) + s(x', t + d, d)\bigr)\), \(x' = x + d\, s(x, t, d)\), which is the semigroup identity for average velocities, with no teacher. On ImageNet at \(256 \times 256\) their largest model reaches \(3.8\) with 128 steps, \(7.8\) with four and \(10.6\) with one.K. Frans, D. Hafner, S. Levine and P. Abbeel, "One step diffusion via shortcut models", ICLR 2025 (arXiv 2410.12557), Section 3 and Table 2 (Shortcut Model XL, 676M parameters, 250 epochs).
Flow map matching. Boffi, Albergo and Vanden-Eijnden compare objectives for learning the full two-time map \(X_{s \to t}\). One version uses a pretrained velocity as a teacher. A later version uses the model's own diagonal \(v_{t,t}\) for self-distillation. They find the Lagrangian objective most stable; it avoids spatial derivatives and the need to build targets by composing small steps.
Sabour, Fidler and Kreis scale continuous-time flow-map distillation with autoguidance. Their 280-million-parameter student reaches ImageNet FIDs of \(1.25\) in two steps at \(64 \times 64\) and \(1.87\) in two steps at \(512 \times 512\), improving to \(1.81\) with adversarial fine-tuning.N. M. Boffi, M. S. Albergo and E. Vanden-Eijnden, "How to build a consistency model: learning flow maps via self-distillation", NeurIPS 2025 (arXiv 2505.18825): "the novel class of Lagrangian methods, which avoid both spatial derivatives and bootstrapping from small steps by design, achieve significantly more stable training and higher performance than more standard Eulerian and Progressive schemes". A. Sabour, S. Fidler and K. Kreis, "Align your flow: scaling continuous-time flow map distillation", NeurIPS 2025 (arXiv 2506.14603), Tables 1 and 2: ImageNet \(64 \times 64\) one step \(2.98\), two steps \(1.25\); ImageNet \(512 \times 512\), AYF-S distilled from EDM2-S, one step \(3.32\), two steps \(1.87\), and \(1.92\) and \(1.81\) with the adversarial loss.
| method | the function learned | the condition imposed | teacher |
|---|---|---|---|
| progressive distillation | \(X_{t \to t + 2h}\), one step for two | semigroup, \(X_{t \to t+2h} = X_{t+h \to t+2h} \circ X_{t \to t+h}\) | yes, round by round |
| consistency distillation, training | \(f(x, t) = X_{t \to 0}(x)\) | Eulerian, in discrete form | distillation yes, training no |
| continuous-time consistency, sCM | the same | Eulerian, \(\frac{d}{dt} f(x_t, t) = 0\) | no |
| shortcut models | the average velocity \(u(x, t + d, t)\) | semigroup for average velocities, plus the velocity at \(d = 0\) | no |
| flow map matching, Lagrangian | \(X_{s \to t}\) | Lagrangian, \(\partial_t X_{s \to t} = u_t(X_{s \to t})\) | yes, or the model's own diagonal |
| MeanFlow | the average velocity \(u(x, r, t)\) | Theorem 8.1.4, with the conditional velocity | no |
The models trained for this series are multi-step; the figure of Section 5.4 runs the browser-sized model with as few as five steps of DDIM, which shows what a few-step sampler loses without a learned flow map.
One step from scratch
Can a model learn to generate in one evaluation without first training a diffusion teacher? Results from 2025 and 2026 increasingly answer yes on standard image benchmarks. The methods differ in how they construct a target when no teacher supplies the trajectory. The table records their dates and publication status alongside the reported results.
A naive alternative would pair an independent noise draw with a random training image and regress directly onto the image. Least squares would learn the data mean, since the noise says nothing about which image was chosen. Teacher-free one-step methods need additional structure to avoid that outcome: relations along paths, agreement between step sizes, or a loss that compares whole output distributions.
MeanFlow is Algorithm 8.1.7; trained from scratch on ImageNet at \(256 \times 256\) in latent space, its XL/2 model with 676 million parameters reaches an FID of \(3.43\) in one evaluation, \(2.93\) in two, and \(2.20\) in two with a longer schedule.
Its successor, the improved MeanFlow of Geng, Lu, Wu, Shechtman, Kolter and He, recasts the loss as a regression on the instantaneous velocity through a reparameterization of the network's output and makes the guidance scale an input, and reaches \(1.72\) in one evaluation and \(1.54\) in two.
Zhou, Ermon and Song's inductive moment matching trains a one- or few-step generator by matching the distribution of its outputs at one noise level to the model's own outputs from a higher level, a moment-matching objective rather than a regression, with guarantees of convergence in distribution; it reaches \(1.99\) with eight steps.
Deng, Li, Li, Du and He's drifting models evolve the generator's output law during training by a drifting field that is anti-symmetric in the two laws and vanishes when they agree, and reach \(1.54\) in latent space and \(1.61\) in pixel space in one evaluation.
The pixel-space MeanFlow of Lu et al. reaches \(2.22\) at \(256\) and \(2.48\) at \(512\) pixels without a latent space, and the preprints of February to October 2026 report \(1.52\), \(1.70\), \(1.29\) and \(1.19\) by methods that combine these ideas with distillation from an intermediate model, with a Wasserstein gradient flow objective, or with a reconstruction pre-training followed by distributional post-training.All numbers are FID-50K on class-conditional ImageNet at \(256 \times 256\) as read on 24 October 2026 from the arXiv HTML or PDF of each paper, with classifier-free guidance where the papers use it. Geng et al., "Mean flows" (2025), Tables 2 and 4: 1-NFE \(3.43\) (XL/2, 676M), 2-NFE \(2.93\), and \(2.20\) for XL/2+ at 1000 epochs. Z. Geng, Y. Lu, Z. Wu, E. Shechtman, J. Z. Kolter and K. He, "Improved mean flows: on the challenges of fastforward generative models", CVPR 2026 (arXiv 2512.02012): iMF-XL/2, 610M, 1-NFE \(1.72\), 2-NFE \(1.54\). L. Zhou, S. Ermon and J. Song, "Inductive moment matching", ICML 2025 (arXiv 2503.07565): "1.99 FID using only 8 inference steps"; the PMLR landing page of the paper carries an older abstract naming the method "Moment Matching Self-Distillation" with \(2.13\), while the camera-ready PDF says \(1.99\). M. Deng, H. Li, T. Li, Y. Du and K. He, "Generative modeling via drifting", arXiv 2602.04770 (February 2026, no accepted venue when read): "an FID of 1.54 in latent space and 1.61 in pixel space". Y. Lu, S. Lu, Q. Sun et al., "One-step latent-free image generation with pixel mean flows", ICML 2026 (arXiv 2601.22158): "2.22 FID" at 256 and "2.48 FID" at 512. T. Zhang, C. Liu, J. Chen et al., "FlowConsist", arXiv 2602.06346 (February 2026, no venue): "an FID of 1.52 with only 1 sampling step". L. Dong, R. Guo, G. Bai et al., "Dual-end consistency model", ECCV 2026 (arXiv 2602.10764): "a state-of-the-art FID score of 1.70 in one-step generation". J. Han, P. Li, Q. Guo, R. Xu, S. Ermon and E. J. Candès, "One-step generative modeling via Wasserstein gradient flows", arXiv 2605.11755 (May 2026, an ICLR 2027 submission): "achieving 1.29 FID". W. Park, J. Kim, Y. Du, S. Chen and B. Han, "Are we overthinking one-step generation?", arXiv 2610.20331 (14 October 2026, an ICLR 2027 submission): "a state-of-the-art FID of 1.19 among one-step generators using approximately one-tenth the training compute of prior methods with comparable FID".
| date | method | FID, 1 step | FID, 2 steps | parameters | status on 24 Oct 2026 |
|---|---|---|---|---|---|
| Oct 2024 | shortcut models, Frans et al. | 10.6 | 676M | ICLR 2025 | |
| Mar 2025 | inductive moment matching, Zhou et al. | 1.99 at 8 steps | ICML 2025 | ||
| May 2025 | MeanFlow, Geng et al. | 3.43 | 2.93; 2.20 at 1000 epochs | 676M | NeurIPS 2025 |
| Oct 2025 | equilibrium matching, Wang and Du | 1.90, by gradient descent on a learned energy, steps not stated | preprint | ||
| Dec 2025 | improved MeanFlow, Geng et al. | 1.72 | 1.54 | 610M | CVPR 2026 |
| Jan 2026 | pixel MeanFlow, Lu et al. | 2.22, in pixels | ICML 2026 | ||
| Feb 2026 | drifting models, Deng et al. | 1.54 latent, 1.61 pixels | 463M | preprint | |
| Feb 2026 | FlowConsist, Zhang et al. | 1.52 | preprint | ||
| Feb 2026 | dual-end consistency model, Dong et al. | 1.70 | ECCV 2026 | ||
| May 2026 | W-Flow, Han et al. | 1.29 | under review | ||
| Oct 2026 | priming models, Park et al. | 1.19 | under review |
Remark 8.3.1 (takeaway: what a one-step objective specifies). Flow-map methods learn the endpoint of a chosen transport, either directly or through its average velocity. Their targets can come from a teacher's trajectory, the model's derivative, or the composition of shorter steps. Distribution-matching methods impose a different requirement: the collection of outputs should have the desired law. They need not reproduce the same endpoint for each noise input, or the trajectories of an existing diffusion model.
These distinctions matter when comparing results. Similar FIDs support a comparison of output distributions under that metric; they do not establish that two generators learned the same map. Equilibrium matching differs again: it learns a time-independent field and samples by gradient descent on its energy, with a step count chosen at sampling time. The table records substantial progress in output quality with few evaluations, while the reliability and cost of learning these different objects remain separate questions.
What a one-step number means, and the difficulty underneath
The table's numbers need context. FID fits Gaussians to the Inception features of 50,000 generated samples and the reference training set, then compares those Gaussians by Fréchet distance. It reflects both fidelity and coverage, but misses some failures that a person can see immediately. At values below about \(2\) on ImageNet, evaluation protocols, reference statistics, and guidance choices can materially affect the comparison.
Step counts also need interpretation. The one-step results incorporate guidance into a single evaluation, whereas a guided multi-step model typically makes two network calls per step. And "from scratch" means without a pretrained diffusion teacher. It can still include pretrained encoders, latent spaces, or feature extractors used in the loss.
Remark 8.4.1 (warning: the fastforward difficulty). Every teacher-free method of Section 8.3 has a target that depends on the model being trained: the derivative in Theorem 8.1.4, the composition in the semigroup condition, the model's own outputs at another noise level in moment matching. This is unlike the regressions of Sections 2 and 6, whose targets, the clean point, the noise, the conditional velocity, are fixed data. Geng et al. call the resulting models fastforward and name the difficulty: the regression is a fixed-point problem in disguise, its loss landscape depends on where the model is, and the stop-gradient that makes it look like a regression means the update need not be the gradient of a single fixed objective. The stabilizations of this part, the tangent normalization and warm-up of sCM, the Lagrangian objective of Boffi et al., the reparameterization of the improved MeanFlow and the reconstruction pre-training of the priming models, are all responses to it, and a theory that says when such a training converges is among the open problems of Section 10.4.Geng et al., "Improved mean flows" (2026), whose subtitle is "On the challenges of fastforward generative models" and whose first section states the problem in these terms; Lu and Song (2025), Section 4, for the instabilities of the Jacobian–vector product term and their remedies.