How AI Generates Images: Diffusion Models Explained

Type "a cat astronaut floating over Mumbai, oil painting" and seconds later a brand-new image appears that has never existed before. Tools like DALL·E, Midjourney and Stable Diffusion feel like pure magic — but underneath almost all of them is one surprisingly elegant idea: diffusion. Having covered how AI understands and generates text (Lesson 20), this lesson crosses into images, and the mechanism is genuinely delightful once it clicks.

The core idea: start with noise, then remove it

Here's the twist that surprises everyone. An image generator doesn't "draw" from a blank canvas. It starts with a square of pure random static — like an untuned TV — and then removes noise, step by step, until a clear image emerges. It's less like painting and more like sculpting: revealing a picture hidden inside the noise.

How it learns: add noise, then reverse it

How does a model learn to remove noise? By practising the forward direction first. During training it takes millions of real images and repeatedly adds a little random noise until each is pure static (the forward process). At every step it records exactly what noise was added. The neural network's job is then simple to state: look at a noisy image and predict the noise. Subtract that prediction and you've taken one step back toward a clean picture.

To generate a new image, you just run this in reverse: start from fresh random noise and let the trained network remove a bit of predicted noise, over and over (the reverse process). Because you began from random noise, you get a brand-new image every time.

See forward diffusion in NumPy (runnable)

The forward "add noise" process — the thing the model learns to undo — is just a few lines. Here we take a simple clean signal and watch it dissolve into noise over several steps.

import numpy as np
np.random.seed(0)

# A tiny 1D 'image': a clean smooth pattern
image = np.array([0.0, 0.3, 0.7, 1.0, 0.7, 0.3, 0.0, -0.3])

x = image.copy()
for step in range(1, 6):
    noise = np.random.randn(len(x)) * 0.3   # add a little noise
    x = x + noise
    print(f"step {step}:", np.round(x, 2))

# After enough steps the pattern is gone -> pure noise.
# A diffusion model is trained to REVERSE each of these steps.

Run it and watch the smooth ramp turn to static. The genius of diffusion is training a network to walk that ladder backwards — from static back to a clean, meaningful image.

How your text prompt controls the picture

Pure diffusion gives a random image. To make "a cat astronaut" appear, the denoising is guided by your text. A model like CLIP converts your prompt into a vector that captures its meaning, and the denoiser is conditioned on it — at every step it's nudged toward noise-removal that matches "cat astronaut." That's why the same prompt gives different-but-on-theme images: same guidance, different starting noise.

Prompting for images (practical tips)

• Be descriptive: subject, setting, lighting, style, mood — "a red fox in a snowy forest, golden hour, cinematic."
• Name a style: "watercolour," "3D render," "pixel art," "photorealistic."
• Add quality/format cues: "highly detailed," "wide-angle," aspect ratio.
• Iterate like Lesson 21 — adjust and regenerate; small wording changes matter a lot.

A quick note on GANs

Before diffusion took over, most image generation used GANs (Generative Adversarial Networks) — two networks, a "generator" and a "critic," competing until the fakes look real. GANs are still used, but diffusion models generally produce more diverse, higher-quality images and now power the leading tools.

Uses — and the ethics you must know

Image generation is powerful for design, marketing, prototyping, art and education. But it raises real, serious issues you should understand as a responsible practitioner:

Use these tools to create, not to deceive — and we'll dig into responsible AI properly in the next lesson.

Remember this: an AI image generator doesn't paint — it de-noises. It starts from random static and removes noise, step by step, guided by your words, until a picture appears.

Diffusion stayed abstract for me until I ran that little noising loop and watched a clean pattern crumble into static — realising the model just learns to play that tape backwards is the moment image generation stopped feeling like magic and started feeling like a mechanism I could reason about. Try nudging the noise amount up and down in the snippet; seeing structure survive or vanish makes the idea stick.

Key takeaways

• Diffusion models (DALL·E, Midjourney, Stable Diffusion) generate images by removing noise, not by drawing.
• Training adds noise to real images (forward) and teaches a network to predict and remove it (reverse).
• Generation starts from random noise, so every result is new.
• Your text prompt guides the denoising (via models like CLIP), turning words into images.
• The tech is powerful but carries real ethical risks — deepfakes, style/copyright, bias and consent.

Continue the series: ← Lesson 23: AI Agents  ·  Next: Lesson 25 — AI Ethics, Bias and Responsible AI →