Derive multipliers from tangent directions

Try

Minimize x²+y² subject to x+y=4. Substituting y=4−x gives 2(x−2)²+8, so the unique minimum is at (2,2), with value eight. At that point the objective gradient is (4,4), not zero. It is perpendicular to the feasible line. Constrained stationarity means no first-order change in feasible tangent directions, rather than no first-order change in every direction of the surrounding plane.

Predict

Predict how to express perpendicularity without eliminating variables. The equality h=x+y−4 has gradient (1,1), a normal to its level set. The objective gradient must be a multiple of this normal. Using L=f+λh, stationarity is ∇f+λ∇h=0, yielding λ=−4. If the equality is written as 4−x−y=0, the multiplier changes sign. Its sign cannot be interpreted without the chosen constraint convention.

Experiment

Inspect several feasible tangent directions at the candidate point. Every tangent d satisfies d₁+d₂=0.

Direction d	Constraint derivative ∇h·d	Objective derivative ∇f·d
(1,−1)	0	0
(−2,2)	0	0
(1,0)	1	4

The final direction is not tangent, so its nonzero objective derivative does not refute constrained optimality. Trace a small feasible displacement (2+t,2−t); its objective is 8+2t². This supplies a direct second-order and global check for this example, stronger than stationarity alone.

Understand

Let h:Rⁿ→Rᵐ define smooth equalities h(x)=0. If Dh(x⋆) has full row rank, the implicit-function theorem makes the nearby feasible set a smooth surface with tangent space ker Dh(x⋆). Every tangent vector can be realized as the initial velocity of a feasible curve. At a constrained local minimum, differentiating f along such curves gives ∇f(x⋆)·d=0 for every tangent d. Linear algebra identifies the orthogonal complement of ker Dh with the image of Dhᵀ, so there is a multiplier λ satisfying ∇f+Dhᵀλ=0.

This is a necessary condition under the stated regularity, not a sufficient optimality test. The equality multipliers are unrestricted real numbers. Full row rank also makes the multiplier unique for a given stationary point, since Dhᵀ is injective. Finding several solutions to the stationarity equations requires classifying and comparing them; an algebraic solver may return constrained maxima and saddles as well as minima.

Apply

Examine min f(x)=x subject to h(x)=x²=0. The only feasible point is zero, so it is automatically a global minimizer. But the Lagrange equation would be 1+λ·0=0, which has no solution. The constraint gradient vanishes and full rank fails. The failure is not a contradiction: the multiplier theorem’s hypothesis was absent. Replacing the equality by x=0 describes the same feasible set with a regular representation and yields a multiplier. Constraint representation can therefore affect qualification checks even when the feasible points coincide.

For a general affine equality aᵀx=b with a≠0, the minimum-norm point is x⋆=ba/||a||². Any feasible x=x⋆+d has aᵀd=0, giving ||x||²=||x⋆||²+||d||². This proves a unique global optimum directly. Use the multiplier calculation to find a candidate and the orthogonal decomposition to certify it; the two arguments have different roles.

Continue

State differentiability, rank, tangent space and multiplier convention before applying Lagrange equations. Continue to KKT and constraint qualifications, where one-sided feasible directions introduce nonnegative inequality multipliers and complementary slackness. The distinction between necessary conditions and sufficient convex certificates will remain essential.