Find the input and output axes of any linear map

Try

Apply A=diag(3,1) to points (cos θ,sin θ) on the unit circle. Their outputs are (3cos θ,sin θ), an ellipse with semiaxis lengths three and one. The input axes and output axes happen to agree here. For a general map they need not agree, and for a rectangular map they do not even lie in the same space. Singular-value decomposition identifies both sets of orthonormal axes and the stretch between them.

Predict

Predict the ellipse produced by [[0,1],[3,0]]. The first input axis maps to an output vector of length three in the vertical direction; the second maps to a horizontal vector of length one. Its singular values are still three and one, but its input/output pairing differs from the diagonal example. Eigenvalues answer a different question: they require input and output to be compared in the same vector space and seek directions mapped to scalar multiples of themselves.

Experiment

Live experiment: rotate, stretch and recover

Construct A=UΣVᵀ with two rotations U,V and Σ=diag(σ₁,σ₂). The unit circle becomes an ellipse: right singular directions v₁,v₂ enter the map, and left directions u₁,u₂ orient its output axes. Predict which controls change the ellipse and which change where a particular input lands.

First singular value σ₁ 123Second singular value σ₂ 00.250.51Output rotation θ 0°30°45°60°90°120°150°180°Input-basis rotation φ 0°30°45°60°90°120°150°180°Input direction α 0°30°45°60°90°120°150°180°225°270°315°Reset experiment

A has rows [1.732051, -0.25] and [1, 0.433013]. The input x=(0.707107, 0.707107) maps to Ax=(1.047968, 1.013293), with length 1.457738.

v₁=(1, 0) maps to σ₁u₁=(1.732051, 1); v₂=(0, 1) maps to σ₂u₂=(-0.25, 0.433013). The operator norm is σ₁=2. Rank is 2 and the full-space 2-norm condition number is 4.

Apply the pseudoinverse A⁺=VΣ⁺Uᵀ to Ax: recovered input (0.707107, 0.707107), with error length 0. Σ⁺ reciprocates positive singular values and leaves zero at zero. For this full-rank map, A⁺ is the inverse and recovers x up to browser roundoff. A small σ₂ amplifies perturbations in the corresponding inverse direction.

Keeping only σ₁u₁v₁ᵀ produces a rank-one map. Its error on v₂ has length σ₂=0.5; the entire error operator has norm σ₂ because it is σ₂u₂v₂ᵀ. A displayed ellipse illustrates this identity, while the SVD theorem and its proof explain when such a decomposition exists generally.

Set σ₁=σ₂=1, θ=90°, φ=0°. A is a quarter-turn: its real singular values are 1,1 although its complex eigenvalues are i and −i. Then set σ₂=0 and α=90°,φ=0° to lose the input completely. This real 2×2 experiment is not a proof of the general real/complex spectral theorem. Curves use 180 line segments; settings are temporary.

Use the controls to rotate the input and output axes independently. Compare the chosen input vector with its image, then set the smaller stretch to zero and inspect the pseudoinverse reconstruction. The controls are temporary exploration, not saved assessment evidence.

The following is a separate fixed worked example for A=[[0,1],[3,0]], not the widget’s current settings. Build this axis table and verify each matrix-vector product. Then replace the smaller stretch by zero and describe the image of the entire circle: it becomes a line segment, not a smaller nondegenerate ellipse.

Input direction	Output	Length ratio
(1,0)	(0,3)	3
(0,1)	(1,0)	1
(1,1)/√2	(1,3)/√2	√5

The largest and smallest axis stretches bound all unit-vector output lengths in this invertible example.

Understand

For A:Fⁿ→Fᵐ, with F equal to R or C and standard inner products, A*A is self-adjoint positive semidefinite. Choose an orthonormal eigenbasis vⱼ with eigenvalues σⱼ²≥0. Order the positive σⱼ decreasingly. For σⱼ>0 define uⱼ=Avⱼ/σⱼ. The identity ⟨Avᵢ,Avⱼ⟩=⟨vᵢ,A*Avⱼ⟩ proves these uⱼ are orthonormal. Complete them to an orthonormal basis of the output space. Zero eigenvalues correspond exactly to ker A, so their vectors map to zero. In these two bases A has a rectangular diagonal matrix Σ with nonnegative entries, yielding A=UΣV*. This proves existence without assuming A is square or diagonalizable.

Apply

The positive singular values are uniquely determined as square roots of the positive eigenvalues of A*A, including multiplicities. Singular vectors within repeated-value subspaces are not unique, and phases can be changed consistently. Expanding x in the v-basis gives ‖Ax‖²=Σσⱼ²|cⱼ|², proving ‖A‖₂=σ₁ and rank A equal to the number of positive singular values. The Frobenius norm squared, the sum of squared matrix entries, equals Σσⱼ²: it is tr(A*A), unchanged by the unitary eigenbasis. For a normal square matrix, singular values are eigenvalue magnitudes; for a general matrix that equality need not hold. A nonzero nilpotent matrix is a decisive counterexample.

Continue

SVD supplies stable geometric coordinates for approximation, least squares and conditioning. It is a theorem about exact matrices; numerical rank still requires a scale-dependent tolerance because tiny nonzero singular values may be indistinguishable from noise. The next lesson uses reciprocal positive singular values to build a pseudoinverse, while leaving zero directions unresolved rather than pretending to reconstruct lost information. The widget, when present, is a two-dimensional illustration of these finite-dimensional statements.

Review Linear Algebra I when a prerequisite needs practice. Continue to Solve least squares while exposing lost directions.