① Intuition
DDPM and LDM showed that diffusion models can generate stunning images. But left unconditioned, they sample from the full data distribution — there is no way to say “generate a photo of a red fox in snow.” Conditioning on a text prompt or class label is essential. The obvious approach: train an auxiliary classifier on noisy images at every noise level t, and at inference time use its gradient to steer the denoising trajectory toward the desired class. This is classifier guidance (Dhariwal and Nichol, 2021). It works — but it requires training a second model (the noisy-image classifier), which is expensive and tightly coupled to the specific noise schedule of the diffusion model.
Ho and Salimans (2022) asked: can we guide without any classifier at all? The answer is yes. The key insight: a single model can implicitly encode both a conditional score and an unconditional score. During training, the conditioning signal c (e.g. a class label or text embedding) is randomly replaced with a null token ∅ with probability p_uncond (typically ~10%). The network learns to denoise with the condition when it is present, and without it when it is absent. At inference time, you run the network twice per denoising step — once with c and once with ∅ — then combine the two noise predictions with a guidance weight w. Cranking up w trades diversity for sharpness: the generated distribution shrinks toward the conditional mode.
Before using the widget: if w=0 in the CFG formula, what do you expect the guided distribution to look like compared to the conditional and unconditional distributions?
At w=0 the formula reduces to the pure conditional noise prediction — the guided distribution is exactly the conditional distribution. There is no extrapolation away from it; the unconditional prediction is multiplied by zero and cancels out entirely.
Interact with the widget: drag the guidance scale slider from 0 to 10. The orange curve is the guided density. Toggle “Show both source distributions” to reveal the conditional (blue, narrower peak) and unconditional (grey, wider and shifted left). Watch how increasing w sharpens the orange curve and pulls its mode toward the conditional mode.
Classifier-Free Guidance
After playing with the widget at high w (e.g. w=10): the mode of the guided distribution moves past the conditional mode and the tails narrow dramatically. What does this tell you about the cost of very high guidance?
Very high guidance extrapolates aggressively along the direction from the unconditional mode to the conditional mode, moving the effective sample distribution beyond the conditional mode itself. This sharpens the density (reducing diversity) but can push samples into low-probability regions of the true data distribution — causing artifacts or over-saturated, mode-collapsed outputs. In practice w=7.5 is a common sweet spot for text-to-image models.
Retrieval check
Answer all 2 correctly to mark this stage mastered.
② Math
The CFG inference formula combines the conditional and unconditional noise predictions with guidance weight w:
In score-function form this is equivalent to:
The formula linearly extrapolates from the unconditional score toward the conditional score. When w=0 you recover the pure conditional score. When w=1 you get 2conditional - 1unconditional, pushing further toward the conditional mode. The training loss is the standard DDPM objective applied jointly:
where c_drop equals c with probability (1 - p_uncond) and equals ∅ with probability p_uncond. The network sees both branches during training and learns to handle both. This is the entire training change — no new loss terms, no new model.
Show next line
Show next line
Retrieval check
Answer all 2 correctly to mark this stage mastered.
③ Pseudo code
Arrange the four steps of a single CFG denoising step in the correct order:
- Apply standard DDPM update rule using e_guided to obtain x_{t-1}
- Combine with guidance weight: e_guided = (1 + w) * e_cond - w * e_uncond
- Compute unconditional noise prediction: e_uncond = eps_theta(x_t, null)
- Compute conditional noise prediction: e_cond = eps_theta(x_t, c)
Fill in the blank — the CFG combination step:
def cfg_combine(e_cond, e_uncond, w):
# e_cond, e_uncond: noise predictions, same shape
# w: guidance scale (>=0)
e_guided = ______
return e_guidedRetrieval check
Answer all 1 correctly to mark this stage mastered.
④ Code (Python)
Implement the CFG combination in NumPy. Given arrays e_cond and e_uncond (noise predictions of the same shape), and a guidance scale w, compute e_guided. Then verify that w=0 returns the conditional prediction and that w>0 shifts the result in the direction of e_cond - e_uncond.
Retrieval check
Answer all 1 correctly to mark this stage mastered.
⑤ Original paper
Suggested reading order: §1 (Introduction — motivation for avoiding a classifier and the dropout trick) → §2 (Classifier-Free Guidance — the training procedure and inference formula) → §4 (Experiments — FID vs. IS tradeoff curves showing the effect of varying w).