Two equally frequent groups have conditional means 2 and 8, each with conditional variance 3. The overall mean is 5. Predict why averaging the two variances gives too small an overall spread: the group centers themselves differ from 5.
Predict
Imagine replacing every observation by its group’s mean. Within-group variation disappears, but the values 2 and 8 still vary. Now imagine centering each observation at its group mean. Those residuals retain within-group variation while their conditional means become zero. These two pieces explain total variance.
Experiment
The average conditional variance is 3. The variance of conditional means is [(2−5)²+(8−5)²]/2=9. Total variance is therefore 12.
Component
Quantity
Value
Within groups
E[Var(X∣Y)]
3
Between group means
Var(E[X∣Y])
9
Total
Var(X)
12
If you predict X using the correct group mean, the remaining mean squared error is 3. Predicting the constant 5 instead has error 12. The extra nine is the cost of ignoring informative group membership under this squared-error criterion.
Understand
For square-integrable X, let m(Y)=E(X|Y). Split X−E(X) into (X−m(Y))+(m(Y)−E(X)). Squaring produces two squared terms and a cross term. Conditional on Y, the first residual has mean zero, while the second factor is known. Thus the cross expectation vanishes.
The resulting identity is Var(X)=E[Var(X|Y)]+Var(E[X|Y]). Both components are nonnegative. The first averages uncertainty remaining within conditional laws; the second measures how their centers move. Reversing the words “expectation of variance” and “variance of expectation” changes the quantity, so retain the parentheses.
For any square-integrable predictor g(Y), a similar decomposition gives E[(X−g(Y))²]=E[(X−m(Y))²]+E[(m(Y)−g(Y))²]. The conditional mean minimizes expected squared error, with uniqueness up to almost-sure equality. This is a mathematical optimality claim within the chosen information and loss, not a guarantee about an estimated model trained on limited data.
The proof requires finite second moments so the squared terms and orthogonality argument are meaningful. It also requires that the predictor use only the information in Y. A function with access to additional information solves a different prediction problem. Adding information cannot increase the minimum achievable squared error, though fitting an unreliable empirical model can perform worse in practice.
A conditional median minimizes an absolute-error criterion under suitable conditions, illustrating that the preferred predictor depends on loss. Do not describe a conditional mean as universally best without naming squared error. Costs that treat underprediction and overprediction differently may call for another decision rule.
Total variance can reveal hidden group composition. A population with individually consistent groups can still have large total spread when group means differ. Conversely, similar group means do not imply small within-group variability or identical conditional distributions. Report both mechanisms where they matter.
Apply
Derive the zero cross term by conditioning rather than simply declaring the residuals independent. They need not be independent; conditional mean zero is enough. For the practice mixture, show both variance components and check that their units match. Then compare two predictors through their excess-MSE term.
Continue
Next derive laws of sums and multivariable transformations. The covariance identity describes only moments; a full sum distribution needs the joint law and correct support.
Apply · explain · revise
Practice and reasoning
Drafts save locally as you type. Reveal worked solutions when useful; the site remembers that exposure. Automatic matches check the requested answer, while written arguments require review against the rubric.
E[Var(X|Y)]=3 and Var(E[X|Y])=2. Find Var(X).
Integers, fractions and finite decimals are accepted. Expressions such as sqrt(2) are not parsed by this checker.
Equally likely groups have conditional means 1 and 5 and conditional variances 2 each. Find total variance.
Integers, fractions and finite decimals are accepted. Expressions such as sqrt(2) are not parsed by this checker.
Averaging only conditional variances generally omits:
For squared-error loss and finite E(X²), the best predictor using Y is:
Derive total variance using m(Y)=E(X|Y).
Structure your argument and preserve revisions
These optional planning fields save locally. Recording your response saves an immutable copy of the argument and these fields. Older records may contain only first/latest responses; missing intermediate revisions are not reconstructed.
Each field allows 2,000 characters. Keep a specific claim, state where it applies, cite the facts you use, and explain what still needs checking. Blank fields are allowed and are not evidence of mastery.
For square-integrable X and predictor g(Y) with finite second moment, prove the MSE decomposition around m(Y)=E(X|Y).
Structure your argument and preserve revisions
These optional planning fields save locally. Recording your response saves an immutable copy of the argument and these fields. Older records may contain only first/latest responses; missing intermediate revisions are not reconstructed.
Each field allows 2,000 characters. Keep a specific claim, state where it applies, cite the facts you use, and explain what still needs checking. Blank fields are allowed and are not evidence of mastery.
Reflect and continue
Before marking this lesson complete, explain one key idea without the worked solution and identify a condition that matters. Marking complete records participation, not independent mastery.
