Ask which nearby problem a computation solves

Try

Compute √(10001)−100 with a calculator retaining only a few digits. The true answer is about 0.004999875, so subtracting rounded values near 100 can erase much of it. Rationalization gives the identical expression 1/(√(10001)+100), which adds positive quantities instead. The mathematical problem has not changed, but the numerical path has improved. Verify the algebraic identity before interpreting any observed accuracy difference.

Predict

Predict whether every cancellation is avoidable. When the data are two nearly equal uncertain measurements, their difference may be exactly the question and its sensitivity is intrinsic. In the square-root example, subtraction is merely one algebraic route to a positive answer. A stable reformulation avoids unnecessary cancellation in intermediates. This distinction prevents a misleading conclusion that subtraction itself is always numerically forbidden.

Experiment

Compare the formulas using toy rounded roots.

Rounded root	Direct difference	Rationalized result using that root
100.00	0	1/200
100.01	0.01	1/200.01

Both final-column values remain near the intended result. They are still rounded approximations, not exact answers. Calculate their relative errors using a sufficiently accurate independent reference and explain why the direct differences vary so dramatically.

Understand

Forward error compares computed x̂ with exact x. Backward error asks for an input perturbation making x̂ exact. In Ax=b, residual r=b−Ax̂ means x̂ solves Ax̂=b−r exactly. Thus ‖r‖/‖b‖ is a right-hand-side-only relative backward error when b≠0. Allowing changes to A gives a different model. For invertible A, the error equation x−x̂=A⁻¹r gives relative forward error at most κ(A)‖r‖/‖b‖ in compatible norms. Small backward error and moderate conditioning together imply accuracy; backward stability by itself does not overcome ill-conditioning.

Apply

A backward-stable algorithm solves a nearby problem with changes comparable to arithmetic precision, under its stated assumptions. For summation, a common bound scales with unit roundoff times the sum of absolute terms. The relative error can be large if the exact sum nearly cancels. Pairwise summation reduces accumulation depth; compensated summation tracks lost low-order contributions. Neither changes the uncertainty already in the data. For quadratic roots, choose a numerator that adds like-signed magnitudes, then obtain the other root from their product. Zero denominators, degenerate coefficients and overflow need separate handling rather than being hidden inside a formula.

A certificate must name the allowed perturbation

A small residual is a statement about a nearby problem, not automatically about a nearby solution. Take A=diag(1,ε), b=(1,ε) and candidate x̂=(1,0), where ε=10⁻⁶. The exact solution is x=(1,1). In the infinity norm the relative forward error is 1, while r=b−Ax̂=(0,ε) gives RHS-only relative backward error ε. The matrix condition number is 1/ε, so the bound κ(A)ε=1 correctly permits the large error.

The componentwise RHS-only error is different: it is the largest ratio |rᵢ|/|bᵢ|, with a zero denominator requiring zero residual in that component. Here it equals 1, not ε. Scaling a small equation should not silently turn a completely wrong component into an accurate component. Multiplying the second equation by 1/ε makes the normwise RHS residual 1, although the exact equations and candidate solution have not changed.

Worked perturbation comparison. If changes to b are allowed, replacing its second component by zero makes x̂ exact. If only the diagonal entries of A may change and b is fixed, no perturbation works: the second diagonal entry always multiplies x̂₂=0. If arbitrary entries of A may change, setting the (2,1) entry to ε does work. Thus “the backward error” is incomplete unless the permissible data changes and their scaling are stated.

Changed written task. Use candidate (1,1/2). Compute residual, normwise RHS backward error, componentwise RHS error and forward error. Feedback: r=(0,ε/2), giving ε/2, 1/2 and 1/2 respectively. The normwise condition bound is sharp in this family. The example is exact arithmetic; rounding in residual evaluation would require a separate error bound. Before trusting an implemented linear solver, test zero and tiny right-hand-side components and report both the perturbation model and units.

Predict, change the case, and repair the claim

Case Small residual, large errorChanged perturbation model

A=diag(1,10⁻⁶), b=(1,10⁻⁶), candidate x̂=(1,0).

What is the relative infinity-norm forward error?

10⁻⁶ 1 0

Check this claim Reset practice

Before checking, write the hypothesis or calculation that decides your answer. Change cases and explain which assumption changed. These fixed cases are public guided practice, not a saved score or independent proof assessment.

Continue

Verification needs exact small cases, independent references, residuals and adversarial cancellation tests. Agreement between two implementations of the same unstable formula is weak evidence. Label each statement as an exact identity, an error bound, a first-order approximation or an empirical observation. Later chapters reuse this framework for root finding, linear systems and ODEs. Written implementation prompts specify algorithms and checks; they do not run code or certify a library. A useful report includes the perturbation model and termination reason alongside the final digits, so another reader can assess what those digits establish.

Review calculus and linear algebra when a prerequisite needs practice. Continue to Turn a sign bracket into a certified root interval.