Send the error backward.

This is the same reference used in Technical mode. Expand it whenever you need the exact numbers, assumptions or further explanation.

First, keep the forward values

We compute h = w₁x, prediction = w₂h, and L = ½(prediction − target)². The factor ½ makes the derivative simpler; it is a different scaling from the previous lesson, so compare losses within this example.

With input 2, target 10, and weights 3 and 1, hidden value h = 6, prediction = 6, and loss = 8. Keep these intermediate values: the backward pass will reuse them.

Multiply local sensitivities along each path

The derivative of loss with respect to prediction is prediction − target, here −4. Changing w₂ changes prediction by h times as much, so its gradient is −4 × 6 = −24.

For w₁, the influence passes through h and then w₂. Multiply the local sensitivities: −4 × w₂ × x = −4 × 1 × 2 = −8. This is the chain rule: changes propagate through dependent operations.

Now the optimizer can take a step

Use gradients from the same old weights for both updates. With rate 0.01, w₁ becomes 3.08 and w₂ becomes 1.24. The new prediction is 7.6384 and the loss is about 2.7886. Updating one weight before calculating the other gradient would be a different procedure.

Backpropagation reuses intermediate values rather than perturbing every parameter and rerunning the whole model. Automatic differentiation applies these rules across large graphs. Long chains can shrink or amplify gradients; initialization, residual connections, normalization, and gradient clipping can help.