Taylor expansion and quadratic remainders

Try

For f(x,y)=x²+xy+2y² at the origin, the gradient is zero and the Hessian is [[2,1],[1,4]]. The function is exactly one half of the quadratic form hᵀHh, with no higher remainder. Completing the square gives (x+y/2)²+(7/4)y², positive for every nonzero displacement. This concrete example illustrates how Hessian definiteness controls local shape.

Predict

Predict what changes for a nonquadratic function. The Hessian at the base point supplies the leading quadratic term, but a remainder remains. A sign conclusion requires that remainder to be smaller than the quadratic term uniformly across directions. Merely observing positivity on two coordinate axes is not enough to prove a local minimum; mixed directions may behave differently.

Experiment

Reduce the expansion to a line segment.

The bilinear notation D²f[p,q] applies the Hessian to two directions. It is a scalar for a scalar-valued function. Keep the chosen displacement h fixed while differentiating in t; confusing these roles can introduce missing powers of h.

Understand

For a C² scalar function on an open neighborhood of the joining segment, one-variable FTC twice gives φ(1) = φ(0) + φ′(0) + ∫₀¹(1−t)φ″(t)dt. One derivation writes φ′(s) = φ′(0) + ∫₀ˢφ″(t)dt and integrates in s; the triangular integration region supplies the weight 1−t. Substituting the table yields the multivariable integral Taylor formula.

If the Hessian norm is at most M along the segment, the linearization error is at most M||h||²/2. Subtracting the constant Hessian contribution gives the quadratic expansion f(a+h) = f(a) + Df(a)h + (1/2)D²f(a)[h,h] + o(||h||²). Indeed, the remaining error is bounded by one half of ||h||² times the largest Hessian difference from D²f(a) on the shrinking segment. Continuity makes this difference tend to zero.

Why may a C² Hessian be symmetric?

Start with four values at the corners of a small rectangle. Subtract the two edge values from the opposite-corner sum. Dividing by the signed area hk measures how an x-change varies with y. For F(x,y)=xy+x²y+xy² at the origin the quotient is 1+h+k: with h=k=1/4 it is 3/2; halving both steps gives 5/4. Its limit is one. Change the example and aspect ratio before reading the proof.

Here ∂y(∂xF) means first differentiate in x, then in y. Assume F is C² on an open neighborhood of (a,b): its first and second partial derivatives exist and are continuous there. Choose nonzero h and k with the whole rectangle in that neighborhood, and write

Apply the one-variable mean value theorem to x ↦ F(x,b+k)−F(x,b). For some ξ between a and a+h, Δ/h = ∂xF(ξ,b+k)−∂xF(ξ,b). Apply it again to y ↦ ∂xF(ξ,y). For some η between b and b+k,

Reverse the applications: first vary y in F(a+h,y)−F(a,y), then vary x in its y derivative. This gives Δ/(hk) = ∂x(∂yF)(ξ′,η′) at possibly different intermediate points in the same rectangle. Both pairs tend to (a,b) as (h,k) tends to (0,0). Continuity of the mixed partials makes the limits equal. Negative steps cause no problem: the MVT uses the interval between endpoints while the quotients retain their signs. Fixing all other coordinates gives the same argument for each coordinate pair in higher dimensions. C² is a sufficient hypothesis, not a necessary one.

Existing second partials do not suffice

Set F(x,y) = xy(x²−y²)/(x²+y²) away from the origin and F(0,0)=0. The bound |F(x,y)| ≤ (x²+y²)/2 proves differentiability at zero with derivative zero. Away from zero, direct differentiation gives

Each absolute value is at most three times the distance to zero. The first partials tend to their zero values at the origin: F is C¹. On the axes, however, ∂xF(0,y)=−y and ∂yF(x,0)=x. Thus ∂y(∂xF)(0,0)=−1, whereas ∂x(∂yF)(0,0)=1. The pure second partials at zero are zero. All second partials exist at that point, but the continuity required above fails.

The rectangle quotient is (h²−k²)/(h²+k²). Along k=h it is zero; along k=2h it is −3/5. There is no joint limit, and neither fixed-aspect calculation replaces the iterated limits defining mixed partials. A second-order Taylor approximation also fails: the axes force both pure quadratic coefficients to zero, and the diagonal forces the cross coefficient to zero, but F(t,2t)=−6t²/5 is not smaller order than the squared distance 5t². An array of pointwise second partials must not replace a justified second derivative.

Apply

At a critical point with positive definite Hessian, compactness of the unit sphere gives c>0 with D²f(a)[h,h]≥c||h||². For sufficiently small h, the remainder magnitude is below c||h||²/4. The quadratic term contributes at least c||h||²/2, leaving a positive increase for every nonzero small h. Thus the point is a strict local minimum. Negative definiteness gives a maximum, and indefiniteness gives directions of both signs.

Repair “zero determinant or zero Hessian means no extremum.” The functions x⁴+y⁴, −x⁴−y⁴ and x⁴−y⁴ have identical zero first and second derivative data at the origin but respectively a minimum, maximum and saddle. The quadratic test is inconclusive in degenerate directions.

Continue

Optional depth extends Taylor expansions to vector-valued functions componentwise and to higher multilinear derivatives. A Lagrange intermediate point may differ between components; the integral remainder avoids falsely asserting one common point for an entire vector identity.

Save a segment-based derivation and a local-minimum proof with a uniform positive constant. Distinguish a local conclusion from a global optimization claim. Next, an invertible first derivative will supply another kind of local control: enough to solve nearby output equations uniquely through a contraction argument. If the base point is not critical, the linear term may dominate the quadratic term and prevent an extremum despite a positive definite Hessian. For example x+x²+y² has positive definite Hessian everywhere but is not minimized at the origin. Locate stationary candidates before applying a second-order classification.