Generate by removing noise.

This is the same reference used in Technical mode. Expand it whenever you need the exact numbers, assumptions or further explanation.

Three different operations

Place points around four centers on a graph. Adding random noise gradually hides their cluster structure. Now consider three separate jobs: corrupt a known clean point, train a model to predict the noise used in that construction, and generate a new point by a specified reverse sampling process. The lesson keeps those jobs visible instead of treating an animation of cleaner points as evidence of learning.

The data are original synthetic coordinates: 32 training points and 32 interleaved but distinct development points. Each group has eight points around each of four centers. The denoiser has 370 parameters: three inputs, two hidden layers of sixteen tanh units and two outputs. It reads two noisy coordinates and normalized time, then predicts a two-number noise vector. This is an actual trained neural network, but it is not an image diffusion model or a pretrained generator.

Construct a noisy example

The forward schedule has 32 steps. At step t, βt (“beta t”) is the newly introduced noise variance. It increases linearly from 0.01 to 0.25. Define αt = 1 − βt, and let ᾱt be the product of α values from step 1 through t. The bar distinguishes cumulative signal retention from one-step retention.

We can sample the corrupted point at any chosen time directly:

Here x₀ is the two-coordinate clean point, xₜ is its noisy version, and ε is a vector of independent standard-normal noise values. I means unit variance in each coordinate with no cross-coordinate covariance. The square roots scale signal and noise so that their variance contributions follow the schedule. The coefficients do not generally add to one.

As a worked example, if ᾱt = 0.64, one clean coordinate is 1.2 and its noise value is −0.5, the noisy coordinate is 0.8 × 1.2 + 0.6 × (−0.5) = 0.66. The live inspector shows this same arithmetic using the actual selected schedule value and a fixed noise vector [0.6, −0.8]. That vector is a transparent teaching choice, not a newly sampled training batch.

Moving the inspector’s time slider reuses one noise vector while comparing marginal corruption levels. It is not a recorded forward Markov trajectory. At the last step, ᾱ32 is small but nonzero, so some input signal remains. Beginning reverse generation from a standard Gaussian therefore includes a finite-schedule approximation. The lesson does not claim that its terminal forward distribution is exactly standard normal.

Train a noise predictor

Because we constructed each training example, we know its clean point, time and added noise. The model receives only the noisy coordinates and time. Its target is the noise vector, not a cluster label or a hidden copy of the clean point.

For N constructed examples, the lab uses mean squared noise error averaged over both coordinates:

ε̂ is predicted noise; ε is the known training noise. Each update samples one time uniformly from 1–32 and one fresh seeded Gaussian noise vector for each training point. Gradients are calculated from the same preceding model, clipped together to norm 5 if needed, and applied by Adam with its moment estimates and bias correction. The export retains those optimizer values rather than suggesting weights alone are a complete training checkpoint.

Monitoring uses separate fixed draws at five noise levels and averages those levels equally. The zero-prediction baseline is scored on those same draws. Development coordinates never enter gradients, but repeatedly inspecting their results makes them development evidence. Changing the learning rate continues the current optimizer; resetting returns to the seeded initial weights and empty moments.

The displayed loss is the simplified noise-prediction objective. It is not the full variational bound, exact marginal likelihood or sample-quality score. At high noise, several clean points could plausibly explain the same observation. Squared-error training favors a conditional average prediction; it cannot promise to recover the particular noise draw or clean point with certainty.

A clean estimate is not the whole sampler

Given a noisy point and predicted noise, rearranging the corruption equation gives an estimate of its clean origin:

The symbol θ stands for learned parameters. If we substitute the exact noise used to create a training example, this rearrangement recovers its clean point, up to numerical rounding. During generation the exact noise and clean origin are unavailable; we substitute the network’s estimate.

At large t, the denominator can be small, amplifying noise-prediction errors. Inspect a one-shot clean estimate and its numerical coordinates before assuming it should land on a target cluster. The plot counts points outside its fixed axes instead of clipping their positions into a plausible-looking result.

Reverse sampling uses a chosen transition rule

Freeze a model and start 24 standard-normal points at t = 32. For each point, ask the frozen denoiser for ε̂ and compute the DDPM noise-prediction reverse mean:

Then sample the next point using xₜ₋₁ = mₜ + σₜ ξ, where ξ is fresh independent standard-normal noise. This lab specifies σₜ² = βt(1 − ᾱt−1)/(1 − ᾱt), with ᾱ0 = 1. This is the forward posterior variance used as a fixed reverse-model variance choice. It does not make an imperfect learned reverse mean exact.

At t = 1, that variance is zero, so the last transition adds no random perturbation. At earlier times it generally does. Iterative sampling is therefore not simply subtracting the same predicted noise vector repeatedly or deterministically moving every point toward the nearest target. The network is called again at each new point and time.

Use the one-step control and inspect the first sample’s receipt: input, predicted noise, reverse mean, noise scale, fresh noise and output. The saved-frame selector reviews earlier states without repeating model calls. A full run has 32 steps for 24 points, or 768 individual point-model predictions. The interface reports this work separately from training updates.

Training the live model after freezing does not change an existing sampling run. Start a new run to use the new weights. All random draws are seeded for repeatability; replaying a seed is not independent new evidence. No animation or background process is necessary to understand the sequence.

Evaluate more than a falling loss

At the final frame, the lab checks how many samples lie within 0.45 units of a target center, how those valid samples are distributed across the four centers, and their mean distance to the nearest center. These are deliberately simple task-specific diagnostics. A sample cloud can have good proximity while omitting a mode, or cover all modes with the wrong frequencies and within-mode distribution.

The target points lie around small rings, so merely falling near a center does not establish that the generator matches the full target distribution. Neither these checks nor the noise loss measures real-image fidelity, memorization, usefulness or provenance. Keep failures visible and compare several sampling seeds before making a broader claim.

For guided practice, compare an untrained and trained model at the same sampling seed and explain one reverse receipt. For implementation practice, reproduce fixed-corruption gradient checks, verify the schedule products and test that the final step adds no random noise. For a decision brief, compare loss, mode counts and outliers and state which additional evaluation would be needed for a proposed use.

Export the model, optimizer, bounded sampling trace and your notes before leaving. These public synthetic experiments are practice evidence. The cumulative generative project, independent artifact assessment, human review and learner pilots remain pending.