① Intuition
Before GANs, deep generative models were hard to train well. Variational autoencoders required explicit likelihood formulations and blurry samples; energy-based models were unwieldy. In 2014, Goodfellow et al. proposed a radically different idea: pit two networks against each other in a minimax game.
The Generator G takes random noise z from a prior p_z (e.g. Gaussian) and tries to produce samples indistinguishable from real data. The Discriminator D looks at a sample and tries to tell real from fake. Each network is trained to defeat the other. At equilibrium, G produces samples so realistic that D can do no better than random guessing — outputting 0.5 for both real and fake inputs.
Before playing with the widget: if the generator improves but the discriminator stays fixed, what do you expect happens to D(fake)?
As the generator improves, the fake samples look more realistic, so a fixed discriminator starts mistaking them for real ones — D(fake) rises toward 0.5. The discriminator must keep re-training to stay ahead, creating the adversarial dynamic.
Drag the training-step slider to watch D(real) and D(fake) oscillate during training. Check “Converged equilibrium” to snap both to 0.5 — the Nash equilibrium state.
GAN Adversarial Dynamics
Discriminator output
Generator
Blue = target (2.0) · Orange = G mean
At convergence both D(real) ≈ 0.5 and D(fake) ≈ 0.5. What does this say about the generator's distribution relative to the real data distribution?
When D outputs 0.5 everywhere, it cannot distinguish the two distributions — meaning the generator’s distribution p_g has converged to the real data distribution p_data. The generator has perfectly learned to mimic real data, and the optimal discriminator is a coin flip everywhere.
Retrieval check
Answer all 2 correctly to mark this stage mastered.
② Math
The GAN objective is a minimax value function over the discriminator D and generator G:
D wants to maximize V (correctly label real as real, fake as fake). G wants to minimize V (fool D into labeling fakes as real). Written compactly:
The first term rewards D for outputting high probabilities on real data. The second term rewards D for outputting low probabilities on generated data (and punishes G for failing to fool D).
Show next line
Show next line
Retrieval check
Answer all 2 correctly to mark this stage mastered.
③ Pseudo code
Arrange the GAN alternating training loop steps in the correct order:
- Sample minibatch of m real examples x^(1)...x^(m) from p_data
- Sample minibatch of m noise vectors z^(1)...z^(m) from p_z
- Update D by ascending its stochastic gradient: ∇_θd [Σ log D(x^(i)) + log(1 - D(G(z^(i))))]
- Sample new minibatch of m noise vectors z^(1)...z^(m) from p_z
- Update G by descending its stochastic gradient: ∇_θg [Σ log(1 - D(G(z^(i))))]
Fill the blank in the discriminator update rule:
# Discriminator loss (to maximise)
d_loss = mean(log(D(x_real)) + log(1 - D(______)))Retrieval check
Answer all 1 correctly to mark this stage mastered.
④ Code (Python)
Implement the discriminator loss for one training step in NumPy. Given arrays real_probs (D’s outputs on real data) and fake_probs (D’s outputs on G’s samples), compute the negative minimax objective that D minimises (i.e. the binary cross-entropy loss):
loss = -mean(log(D(real)) + log(1 - D(fake)))Retrieval check
Answer all 1 correctly to mark this stage mastered.
⑤ Original paper
Suggested reading order: §3 (Adversarial Nets — the minimax formulation) → §4.1 (Global optimality of p_g = p_data — the D* derivation and JSD proof) → §4.2 (Convergence of the algorithm — the alternating training convergence argument).