View conditioning as a contraction and projection

Try

An observer predicts a reading using only its group label. Averaging within the group minimizes squared prediction error. For equally likely values 1 and 5 in one group, predicting three gives squared errors four and four. Predicting four gives errors nine and one, a larger mean. Conditional expectation generalizes this calculation to arbitrary information sigma-algebras.

Predict

Predict whether replacing a random value by its conditional average can increase its expected square. Convexity says no. Predict whether it preserves every nonlinear function of the value. Usually not: averaging before squaring differs from squaring before averaging. Conditional Jensen quantifies the direction of this difference.

Experiment

For X=(1,5,2,6) on four equally likely outcomes, group the first two and last two.

Group	Conditional mean	Mean square within group	Square of conditional mean
First	3	13	9
Second	4	20	16

The gap is four in each group, the conditional variance. Compute the overall residual mean square and compare it with the error from predicting the single unconditional mean 7/2. The finer information removes between-group uncertainty.

Understand

Let φ be a finite convex function on the real line, with X and φ(X) integrable. At each rational point choose a supporting affine line below φ. Such lines exist from the interval of one-sided slopes of a convex function. Their countable supremum equals φ: rational points approach any given real x, and supporting slopes remain bounded on each smaller compact interval inside the domain, so the corresponding line values at x approach φ(x).

Each supporting line a_jx+b_j≤φ(x) yields a_jE(X given G)+b_j≤E(φ(X) given G) almost surely by positivity and linearity. Remove the countable union of exceptional null sets and take the supremum. This proves conditional Jensen: φ(E(X given G))≤E(φ(X) given G) almost surely. Countability matters because an arbitrary union of null sets need not be null.

For φ(x)=abs (x)^p, p≥1, taking expectations gives norm (E(X given G))_p≤norm (X)_p. For p=infinity, conditional positivity applied to the essential bounds gives the analogous contraction. In particular conditional expectation maps L2 into L2.

Write Y=E(X given G). The defining integral identity says E((X−Y)1_A)=0 for every G-event A. Extend this first to bounded G-measurable functions by simple approximation. For an arbitrary G-measurable Z in L2, truncate Z; the truncated functions converge in L2, and Cauchy–Schwarz allows passage to the limit. Therefore E((X−Y)Z)=0. The residual is orthogonal to all G-measurable L2 predictors.

For any such predictor Z, expand X−Z=(X−Y)+(Y−Z). Orthogonality removes the cross term, yielding E(X−Z)²=E(X−Y)²+E(Y−Z)². Thus Y is the unique best predictor up to almost-sure equality. Taking Z=E(X) also gives the total-variance decomposition into average conditional variance and variance of the conditional mean.

Apply

Distinguish the loss function from the information constraint. Conditional means minimize squared loss; conditional medians address absolute loss under suitable definitions. Do not infer that the same predictor optimizes every loss. Also distinguish the mathematical oracle E(X given G) from an estimated model fitted to finite data: estimation adds error not present in this projection identity.

Continue

The density definition, convexity inequality and orthogonal projection are three compatible views of conditional expectation. Use the integral definition for existence and tower properties, Jensen for moment control, and projection for squared prediction error. Each conclusion depends on its stated integrability hypothesis.

Continue to Prove weak and strong averaging results.