M05 · optional contextual support
Follow a gradient through a computation
Connect a derivative, chain factors, partial gradients and a numerical check.
Start with arithmetic, percentages and basic algebra. Open any section directly. Practice and completion here do not change any AI course requirement or the original five-lesson mathematics pilot.
1. Try a concrete case
For prediction y=wx and loss (y−t)²/2, w=2,x=3,t=5 gives y=6, error=1, and derivative with respect to w equal to 1×3=3.
2. Predict before changing a value
Change weight and input separately. Multiply the displayed chain factors and compare with the finite difference. Inspect saturation at larger |wx|. Name the quantity you expect to change and one you expect to stay fixed before using the controls.
3. Experiment
Change one quantity, follow every calculation
- Prediction tanh(wx)
- 0.46212
- Half-squared loss
- 0.10678
- Analytic ∂L/∂w
- 0.36343
- Central finite difference
- 0.36343
Vertical axis: Loss. Fixed scales; 0 points clipped for display. Lines join sampled values to aid reading; they do not create extra observations. Exact numerical values are below.
| Chain factor | Value |
|---|---|
| dL/dprediction | 0.46211715726000974 |
| dprediction/dz | 0.7864477329659274 |
| dz/dw | 1 |
| product | 0.36343099069179363 |
Other variables are held fixed for a partial derivative. The central difference checks one point with step 0.0001; cancellation, saturation or other points require separate diagnosis. Simultaneous updates use gradients from the same checkpoint.
Change weight and input separately. Multiply the displayed chain factors and compare with the finite difference. Inspect saturation at larger |wx|.
Controls save locally as practice. No timers, remote services or GPU are required. Maximum numerical work is a bounded small table or 256 bootstrap resamples.
4. Understand the calculation
Start with a calculation you can inspect
For prediction y=wx and loss (y−t)²/2, w=2,x=3,t=5 gives y=6, error=1, and derivative with respect to w equal to 1×3=3.
Connect a derivative, chain factors, partial gradients and a numerical check. The experiment changes one named quantity at a time and shows the intermediate values, rather than asking you to accept a final score. Before changing a control, write down which value you expect to move and why.
Read the formula in words and symbols
∂L/∂w = (prediction−target) × (1−tanh(wx)²) × x
∂ means vary this input while holding others fixed. The middle factor differentiates tanh; the lab uses tanh rather than the linear worked example. A central check uses [L(w+h)−L(w−h)]/(2h).
A subscript names a coordinate, case or time index; it is not multiplication. A summation sign means add the terms for the stated indices. Parentheses identify which calculation happens first. Carry units through arithmetic: adding incompatible quantities or confusing a rate with a count changes the question.
Connect the visual and the numbers
Change weight and input separately. Multiply the displayed chain factors and compare with the finite difference. Inspect saturation at larger |wx|. The plot uses declared fixed axes, so a new setting does not silently rescale the apparent change. Any clipped values are counted and remain available in the numerical table. Do not infer an exact curve or a new observation from a line drawn between points.
Use it in an AI experiment
These ideas support C03, C05, C06, C10, C14. Use the return link when you entered from an AI lesson; its original section and experiment remain separate. A mathematical support calculation is a way to inspect an assumption, not evidence that a trained model generalizes.
Optional derivation and implementation depth
A gradient collects partial derivatives for every parameter. Shared parameters accumulate all path contributions before updating. Vector-Jacobian products backpropagate scalar objectives without constructing every Jacobian entry. A local finite-difference check can detect some bugs but is not proof of global correctness or stable optimization.
Recompute the worked example with a separate piece of arithmetic or a short program. State the domain and test a boundary case before using the formula elsewhere. Guided numerical practice, independent implementation and human review are different outcomes.
A rate at a point is not a finite change
The derivative is the limiting output change divided by input change as the input step shrinks. For y=w², the finite-step quotient from w to w+h is 2w+h; the derivative is 2w. A gradient-descent update w_new=w−rate×gradient moves opposite the local uphill direction. A large rate can overshoot even with a correct derivative.
For a total derivative, include every route by which a variable changes the output. In a recurrent model, the same parameter affects many time steps; backpropagation through time adds those contributions before updating. The tanh derivative 1−tanh(z)² approaches zero when the output saturates near ±1, explaining the small displayed chain factor.
Optional implementation and research depth
A gradient collects partial derivatives for every parameter. Shared parameters accumulate all path contributions before updating. Vector-Jacobian products backpropagate scalar objectives without constructing every Jacobian entry. A local finite-difference check can detect some bugs but is not proof of global correctness or stable optimization.
Recompute one row independently, then test a boundary case and state which assumption it probes. Passing the supplied practice case does not establish independent implementation proficiency.
5. Apply it to a different case
Practice not yet passed; independent mastery and human review are not claimed.
6. Continue when useful
Completion not recorded. Skipping retains settings and practice; it never awards completion.
Used in C03, C05, C06, C10, C14. Browse every support topic and course connection.