Use geometry to derive sampling laws

Try

Take a two-dimensional standard normal vector Z=(Z₁,Z₂). Rotate it into coordinates U=(Z₁+Z₂)/√2 and V=(Z₁−Z₂)/√2. Both have variance one and covariance zero. More strongly, they are independent standard normals because the transformed joint density remains proportional to exp[−(u²+v²)/2]. This factorization, not zero covariance alone, is the reason independence holds. A geometric change of coordinates has exposed two independent pieces of the experiment.

Predict

Predict the distribution of ZᵀPZ when P projects orthogonally onto a subspace of dimension r. After rotating coordinates to align with that subspace, the expression should be a sum of r independent standard-normal squares. Its distribution is chi-square with r degrees of freedom. The remaining coordinates form an independent complement. This argument will support sums of squares in regression, where different projections represent fitted values, residuals and additional explanatory directions.

Experiment

In the plane, project onto the line spanned by (1,1). The matrices below are symmetric, idempotent and multiply to zero. Verify each property with ordinary matrix multiplication.

Projection	Matrix	Quadratic form
Along the diagonal	P=(1/2)[[1,1],[1,1]]	(Z₁+Z₂)²/2
Perpendicular	Q=(1/2)[[1,−1],[−1,1]]	(Z₁−Z₂)²/2

Each form has a χ²₁ law; they are independent and sum to Z₁²+Z₂², which has χ²₂ law. Identify why arbitrary symmetric matrices would not give the same conclusion. Idempotence forces eigenvalues to be zero or one; otherwise weighted squares appear instead of an ordinary chi-square count.

Understand

For Z∼Nₙ(0,I), an orthogonal transformation preserves the joint standard-normal density because it preserves squared length and has Jacobian magnitude one. A real symmetric idempotent P can be orthogonally diagonalized with r ones and n−r zeros, where r is its rank. This proves ZᵀPZ∼χ²ᵣ. For orthogonal projections P and Q with PQ=0, the cross covariance of PZ and QZ is zero. Their concatenation is jointly Gaussian, so a block-diagonal covariance makes the characteristic function factor. This proves independence even though the projected vectors themselves have singular covariance matrices in the ambient space.

For X∼Nₙ(μ,Σ) with positive-definite covariance, set Z=Σ^(−1/2)(X−μ). The symmetric positive inverse square root exists by the spectral theorem, and Z is standard normal. Consequently (X−μ)ᵀΣ⁻¹(X−μ) has χ²ₙ law when μ and Σ in that expression are the true population values. Replacing them by estimates changes the law; this formula alone does not justify a confidence ellipsoid using an estimated covariance.

Apply

A noncentral warning helps delimit the result. If a standard-variance normal vector has a nonzero mean component in the projection space, its projected quadratic form is no longer central chi-square; the squared mean component produces noncentrality. In a regression test, the null hypothesis is exactly what removes this mean component from the tested extra directions. Geometry and the hypothesis must therefore be checked together.

For a nonnormal counterexample, let U be uniform on [−1,1] and V=U². Symmetry gives EU=EU³=0, so Cov(U,V)=0. Yet V is a deterministic nonconstant function of U and is not independent of it. This example pinpoints the missing Gaussian factorization in a false proof that all uncorrelated projections must be independent.

Continue

Review the distinction between orthogonal vectors, orthogonal projection matrices, uncorrelated random vectors and independent random vectors. Each has a precise role, and only specified assumptions connect them. Continue to Regression and ANOVA, where the design matrix supplies the subspaces. The least-squares algebra will remain valid beyond normal errors, but exact t, chi-square and F calibration will continue to require its stated distributional assumptions.