① Intuition
DDPM generates data by defining a Markov chain through a noise schedule: the forward process adds Gaussian noise over T steps (often 1000), and the reverse process denoises step by step. This works beautifully, but it has two costs. First, sampling requires running many sequential denoising steps, which is slow. Second, the denoising path is curved — the network must predict along a complex, high-curvature trajectory from noise to data. These curves mean the model must take many small steps to stay on track.
Flow Matching (Lipman et al., 2022) takes a radically different route. Instead of a discrete Markov chain, it directly parameterizes a time-dependent vector field v(x, t) that continuously transports particles from a base distribution (standard Gaussian at t=0) to the data distribution (at t=1). During training, the target vector field is computed from a constructed conditional path — specifically, a straight line from a noise sample x_0 to a data sample x_1. Straight paths have constant velocity (x_1 - x_0), which is trivially simple to compute and minimizes kinetic energy. At inference you simply integrate the ODE from t=0 to t=1 — and because the paths are nearly straight, far fewer steps are needed.
Before using the widget: if each particle moves along a straight line from its source position to its target, what happens to the overall distribution as t goes from 0 to 1?
Each particle traces the path x(t) = (1-t)x_0 + tx_1. At t=0 all particles are at their Gaussian source positions; at t=1 they are all at their data-distribution targets. In between, the distribution smoothly interpolates. Because all paths are straight and non-crossing (for a single source-target pair), the marginal distribution p_t flows continuously and without backtracking.
Interact with the widget: drag the time slider from 0 to 1. Watch the blue particles glide in straight lines from their initial Gaussian positions toward the target. Switch the target mode to BiModal to see particles split toward two clusters — still straight-line paths per particle.
Mean: 0.176 Variance: 1.164
After playing with the widget: at t=0.5 the particles are halfway between source and target. What does the vector field v(x, 0.5) look like at each particle location, and why is this simpler than a DDPM denoising step?
At any intermediate t, v(x, t) = x_1 - x_0 — a constant velocity vector pointing directly from each particle’s source to its target. This is the same vector regardless of t, because straight-line paths have constant velocity. In DDPM, the denoising network must predict the full noise residual along a curved, schedule-dependent trajectory, which varies with t. Here the target is a single fixed vector per particle, making the regression problem geometrically simple.
Retrieval check
Answer all 2 correctly to mark this stage mastered.
② Math
The generative model is defined by the probability-flow ODE:
Starting from x_0 ~ N(0, I), integrating this ODE to t=1 produces a sample from the learned distribution. The network v_theta is trained to match the marginal vector field u_t(x), which is the average velocity at each point (x, t) under the marginal probability path p_t.
The flow matching loss is:
The marginal u_t is intractable (it requires integrating over all data points). The key insight of conditional flow matching is that training on individual conditional paths is equivalent: for a straight-line path conditioned on a source-target pair (x_0, x_1), the conditional velocity is simply:
The conditional path is x(t) = (1-t) x_0 + t x_1. Substituting into dx/dt gives the constant vector x_1 - x_0. This makes the regression target constant across all t — an enormous simplification compared to DDPM’s t-varying noise prediction.
Show next line
Show next line
Retrieval check
Answer all 2 correctly to mark this stage mastered.
③ Pseudo code
Arrange the five steps of a single flow-matching training iteration in the correct order:
- Compute MSE loss: L = ||v_theta(x_t, t) - target||^2 and update theta
- Sample data point: x_1 ~ p_data
- Compute interpolation and target velocity: x_t = (1-t)*x_0 + t*x_1; target = x_1 - x_0
- Sample source noise: x_0 ~ N(0, I)
- Sample time: t ~ Uniform(0, 1)
Fill in the blank — the target velocity in the conditional flow matching loss:
def cfm_loss(v_net, x0, x1, t):
x_t = (1 - t) * x0 + t * x1
target = _______
return mean_squared_error(v_net(x_t, t), target)Retrieval check
Answer all 1 correctly to mark this stage mastered.
④ Code (Python)
Implement the flow-matching loss for straight-line conditional paths in NumPy. Given arrays x0 (N, d) and x1 (N, d), and a mock network function v_net(x, t), return the MSE loss between the network output and the conditional velocity target.
Retrieval check
Answer all 1 correctly to mark this stage mastered.
⑤ Original paper
Suggested reading order: §1 (Introduction — motivation and comparison to diffusion score matching) → §3 (Flow Matching — the ODE formulation and the FM objective) → §4 (Conditional Flow Matching — the tractable CFM loss and the equivalence to FM gradients).