Separate a vector into measured and invisible parts

Try

To approximate x=(3,1) by a multiple of u=(1,1), let p=cu. Requiring the residual x−cu to be orthogonal to u gives 4−2c=0, hence c=2 and p=(2,2). The residual (1,−1) has squared length two. Draw the line, x, p and the right angle at p. Then compare p with the alternative (1,1): its squared error is four, and the difference is exactly the squared distance from (1,1) to p.

Predict

Predict why orthogonality gives a global minimum rather than merely a stationary point. For any other u in the subspace, x−u splits into the residual x−p and the within-subspace displacement p−u. These terms are perpendicular, so their squared lengths add. No derivative or numerical search is needed. The same argument works over C when cross terms are paired as conjugates and their real sum vanishes.

Experiment

Choose an orthonormal basis q₁,…,qᵣ of a subspace U and compute p=Σ⟨x,qⱼ⟩qⱼ. Check ⟨x−p,qⱼ⟩=0 for each j. As a complex example let x=(1,2i), q=(1,i)/√2. Then ⟨x,q⟩=3/√2 and p=(3/2,3i/2). The residual (−1/2,i/2) has squared norm 1/2. Verify that the squared norms of p and residual sum to ‖x‖²=5. Record the two components and the three squared lengths in a table.

Understand

Gram–Schmidt provides an orthonormal basis of every finite-dimensional subspace. The displayed formula constructs a decomposition x=p+r with p∈U and r∈U⊥. If two such decompositions exist, their difference lies in both U and U⊥ and therefore has squared norm zero; they are identical. Thus V=U⊕U⊥. The projection P is linear, satisfies P²=P, has image U and kernel U⊥. Its best-approximation property follows from ‖x−u‖²=‖r‖²+‖p−u‖². This proof also shows that an algebraic projection along an arbitrary complement need not be a nearest-point projection.

Apply

The inner product represents every linear functional f uniquely as f(x)=⟨x,y⟩. To construct y, choose an orthonormal basis qⱼ and set y=Σconj(f(qⱼ))qⱼ. Expanding x in this basis verifies the identity. If two representing vectors exist, their difference has zero inner product with every x; choose x equal to that difference to prove uniqueness. Because the inner product is conjugate-linear in its second argument, the correspondence y↦f is conjugate-linear over C, not complex-linear. Over R it is linear. This finite-dimensional Riesz representation connects the algebraic dual to geometry while preserving the distinction between them.

For a contrasting algebraic projection, let P(x,y)=(x+y,0). It satisfies P²=P and projects onto the horizontal axis, but P(0,1)=(1,0) is not the closest horizontal point to (0,1); that closest point is zero. Its kernel is the slanted line x+y=0, not the perpendicular axis. This concrete counterexample shows why the best-approximation proof needs the residual to lie in U⊥, rather than merely requiring idempotence.

Normalize the orthogonalization, then prove uniqueness

For A with columns (1,1) and (1,3), Gram–Schmidt gives q₁=(1,1)/√2 and q₂=(−1,1)/√2. The reconstruction coefficients are r₁₁=√2, r₁₂=2√2 and r₂₂=√2. Flipping q₂ and the second row of R keeps QR unchanged. What additional condition singles out the first answer?

What makes a QR factorization unique?

Second-column base s -1012 Second-column separation t -2-1012 Column signs Positive-diagonal choice when full rankFlip first Q column and R rowFlip second Q column and R rowFlip both columns and rows

A has columns (1,1) and (1,3); det(A)=t=2. Predict whether changing signs preserves reconstruction and whether it preserves positive diagonal entries.

Second Gram–Schmidt residual norm: 1.414214. Rank: 2. The R diagonal is strictly positive.

Object	First row	Second row
A	1.000000, 1.000000	1.000000, 3.000000
Q	0.707107, -0.707107	0.707107, 0.707107
R	1.414214, 2.828427	0, 1.414214
QR	1.000000, 1.000000	1.000000, 3.000000
QᵀQ	1.000000, 0	0, 1.000000
AᵀA	2.000000, 4.000000	4.000000, 10.000000
RᵀR	2.000000, 4.000000	4.000000, 10.000000
U=R⁻¹, so Q=AU	0.707107, -1.414214	0, 0.707107

Maximum reconstruction discrepancy 8.88e-16; orthogonality discrepancy 2.22e-16; Gram discrepancy 1.78e-15. These report floating-point evaluation of proved formulas.

Real sign changes are the simplest phase freedom. For complex factors, any diagonal unitary D gives QD and D*R. Requiring a strictly positive real diagonal removes that freedom under full column rank.

Reset

This fixed 2×2 family is not a general numerical QR solver. State is temporary, workload constant and no mastery is awarded.

Let A be an m-by-n real or complex matrix with independent columns, m≥n. Use the standard Euclidean or Hermitian inner product, with the linear-first convention ⟨v,w⟩=wv. At stage j subtract all earlier orthogonal projections from column aⱼ. Its residual uⱼ is nonzero: a zero residual would place aⱼ in the span of earlier columns. Put qⱼ=uⱼ/||uⱼ||, rᵢⱼ=⟨aⱼ,qᵢ⟩ for i<j, and rⱼⱼ=||uⱼ||. The coefficient rᵢⱼ uses the conjugate transpose of qᵢ in complex coordinates. Orthogonality follows from subtracting every earlier component, and reconstructing each column gives A=QR. The matrix Q satisfies QQ=I, and R is upper triangular with strictly positive real diagonal.

To prove uniqueness, suppose A=Q₁R₁=Q₂R₂ with both such normalizations. Since each R is invertible, Q₂=Q₁W with W=R₁R₂⁻¹ upper triangular. Then I=Q₂Q₂=WW, so W is unitary. An upper-triangular unitary matrix is diagonal: its first column has only its first entry, whose modulus is one; orthogonality with the other columns forces every other entry in its first row to vanish. Apply the same argument to the remaining principal block. Its diagonal entries are therefore phases of modulus one. Here each diagonal entry is the ratio of two positive real diagonal entries of R₁ and R₂, so every entry is one. Hence W=I, Q₁=Q₂ and R₁=R₂.

Without the positive-real diagonal requirement, any diagonal unitary D gives another factorization A=(QD)(D*R). Over the reals these phases reduce to signs. The normalized thin Q is unique; when m>n, an additional choice of orthonormal complement to make a full m-by-m unitary matrix need not be unique.

The triangular map that orthonormalizes a basis

Set U=R⁻¹. It is upper triangular with positive real diagonal and Q=AU. Thus column j of Q is a combination of the first j columns of A. Conversely, any upper-triangular positive-diagonal U making AU have orthonormal columns yields the normalized factorization A=(AU)U⁻¹, so the uniqueness just proved also proves uniqueness of U. This is a change-of-coordinates statement, not an assertion that applying A preserves every length.

Multiplying A=QR by its adjoint gives AA=RR. This connects normalized QR to the positive triangular factor of the Gram matrix, while Q carries the geometry in the original output space. It does not recommend forming normal equations numerically: the later numerical lesson explains the conditioning cost.

Full column rank is essential. For columns (1,1) and (2,2), the second Gram–Schmidt residual is zero and must not be normalized. A factorization with Q*Q=I and all positive diagonal entries in R would make A full column rank, a contradiction. Allowing a zero diagonal permits different orthogonal completions of Q, so normalization no longer supplies the same uniqueness theorem. A failed normalization is a rank diagnosis, not a reason to divide by a tiny or zero value without checking.

For preparation, return to Gram–Schmidt and thin QR. For numerical consequences, continue to least squares and QR. The present proof handles real and complex exact algebra; the finite real inspector alone does not verify that general theorem.

Continue

Best approximation underpins least squares, but the result depends on the selected inner product and subspace. Changing weights changes what “nearest” means. The next lesson uses Riesz representation to define the adjoint, which transfers an operator across an inner product. For review, the foundation unit contains Gram–Schmidt and QR calculations; here the goal is the existence, uniqueness and proof structure behind those algorithms.

Review Linear Algebra I when a prerequisite needs practice. Continue to Move an operator across an inner product.