M07 · optional contextual support
Relate a latent approximation to its evidence
Enumerate KL and ELBO exactly, then inspect numerical integration separately.
Start with arithmetic, percentages and basic algebra. Open any section directly. Practice and completion here do not change any AI course requirement or the original five-lesson mathematics pilot.
1. Try a concrete case
A hidden state is 1 with prior probability 0.5. Observation 1 has likelihood 0.8 in state 1 and 0.2 in state 0. Evidence is 0.5, and the posterior state-1 probability is 0.8.
2. Predict before changing a value
Set q to the exact posterior and inspect the gap. Move q away while holding the prior fixed. Separately increase Euler steps and compare its endpoint with exp(−1). Name the quantity you expect to change and one you expect to stay fixed before using the controls.
3. Experiment
Change one quantity, follow every calculation
- KL(q || prior)
- 0.00000
- ELBO
- -0.91629
- Log evidence
- -0.69315
- Gap = KL(q || posterior)
- 0.22314
- Euler endpoint / exact exp(−1)
- 0.32768 / 0.36788
Vertical axis: State y. Fixed scales; 0 points clipped for display. Lines join sampled values to aid reading; they do not create extra observations. Exact numerical values are below.
| Latent z | Prior | Approximate q | P(observation=1 | z) |
|---|---|---|---|
| 1 | 0.5 | 0.5 | 0.8 |
| 0 | 0.5 | 0.5 | 0.2 |
A two-state latent model is exactly enumerable. ELBO = expected log likelihood − KL; its gap is nonnegative. Separately, Euler solves y′=−y using n steps. Reparameterization for a Gaussian is z=μ+σ ε with fixed ε during a gradient check; it does not remove sampling uncertainty.
Set q to the exact posterior and inspect the gap. Move q away while holding the prior fixed. Separately increase Euler steps and compare its endpoint with exp(−1).
Controls save locally as practice. No timers, remote services or GPU are required. Maximum numerical work is a bounded small table or 256 bootstrap resamples.
4. Understand the calculation
Start with a calculation you can inspect
A hidden state is 1 with prior probability 0.5. Observation 1 has likelihood 0.8 in state 1 and 0.2 in state 0. Evidence is 0.5, and the posterior state-1 probability is 0.8.
Enumerate KL and ELBO exactly, then inspect numerical integration separately. The experiment changes one named quantity at a time and shows the intermediate values, rather than asking you to accept a final score. Before changing a control, write down which value you expect to move and why.
Read the formula in words and symbols
ELBO = Eq[log likelihood] − KL(q||prior); log evidence − ELBO = KL(q||posterior)
q is an approximate probability over hidden states. E means a probability-weighted average. KL sums q log(q/p); it is asymmetric and uses natural-log units. All lab probabilities stay strictly inside zero and one.
A subscript names a coordinate, case or time index; it is not multiplication. A summation sign means add the terms for the stated indices. Parentheses identify which calculation happens first. Carry units through arithmetic: adding incompatible quantities or confusing a rate with a count changes the question.
Connect the visual and the numbers
Set q to the exact posterior and inspect the gap. Move q away while holding the prior fixed. Separately increase Euler steps and compare its endpoint with exp(−1). The plot uses declared fixed axes, so a new setting does not silently rescale the apparent change. Any clipped values are counted and remain available in the numerical table. Do not infer an exact curve or a new observation from a line drawn between points.
Use it in an AI experiment
These ideas support C06, C12, C13. Use the return link when you entered from an AI lesson; its original section and experiment remain separate. A mathematical support calculation is a way to inspect an assumption, not evidence that a trained model generalizes.
Optional derivation and implementation depth
For Gaussian sampling, z=μ+σ ε expresses randomness using fixed base noise ε; gradients can flow through μ and σ for a fixed sample. A vector field specifies local motion, while an integration method constructs finite steps. Neither a variational bound nor an ODE solver automatically proves sample quality or physical consistency.
Recompute the worked example with a separate piece of arithmetic or a short program. State the domain and test a boundary case before using the formula elsewhere. Guided numerical practice, independent implementation and human review are different outcomes.
Optional implementation and research depth
For Gaussian sampling, z=μ+σ ε expresses randomness using fixed base noise ε; gradients can flow through μ and σ for a fixed sample. A vector field specifies local motion, while an integration method constructs finite steps. Neither a variational bound nor an ODE solver automatically proves sample quality or physical consistency.
Recompute one row independently, then test a boundary case and state which assumption it probes. Passing the supplied practice case does not establish independent implementation proficiency.
5. Apply it to a different case
Practice not yet passed; independent mastery and human review are not claimed.
6. Continue when useful
Completion not recorded. Skipping retains settings and practice; it never awards completion.
Used in C06, C12, C13. Browse every support topic and course connection.