Derive the regular large-sample approximation

Try

In an iid Bernoulli sample with an interior true probability p₀, the maximum likelihood estimate is the sample proportion. The central limit theorem gives √n(p̂−p₀) converging in distribution to a centered normal with variance p₀(1−p₀). This variance equals the inverse of the one-observation information. The example is especially transparent because the estimate is a sample mean. The general maximum likelihood argument explains how a similar pattern can arise when no simple average formula is available.

Predict

Predict which features would disrupt the argument: a maximum at a support boundary, two indistinguishable parameter values, an infinite score variance, or a likelihood with several poorly controlled maxima. Each can defeat a different step. A smooth-looking graph at one dataset does not certify any of these population properties. Before invoking normality, identify the true parameter as an interior point and the selected estimator as a consistent maximizer under the model being studied.

Experiment

For an exponential rate λ, the sample log likelihood has score n/λ−ΣXᵢ and negative second derivative n/λ². Its maximizer is 1/X̄. Expand the score around the true rate λ₀ and identify the two pieces needed by a limit theorem.

Piece	Normalization	Proposed regular limit
Score at λ₀	divide by √n	normal variance 1/λ₀²
Negative Hessian near λ₀	divide by n	1/λ₀²
Estimation error	multiply by √n	normal variance λ₀²

Check the last row independently using the delta method idea for the reciprocal of a mean. The two derivations should agree. Their agreement is tied to the exponential model and regularity, not to the word “maximum” alone.

Understand

Here is the regular scalar proof mechanism, stated as a sketch because its stochastic uniformity conditions require care. Let ℓₙ be the sample log likelihood and θ₀ the true interior parameter. An interior maximizer satisfies ℓₙ′(θ̂)=0. A Taylor or integral expansion expresses this as ℓₙ′(θ₀) plus a Hessian near θ₀ times θ̂−θ₀. If the normalized score obeys a CLT with variance I₁(θ₀), and the negative normalized Hessian converges uniformly on a neighborhood to a deterministic function J(θ) continuous at θ₀, with J(θ₀)=I₁(θ₀) finite and positive, consistency permits evaluation at the random estimate. Slutsky then gives √n(θ̂−θ₀) converging to N(0,I₁⁻¹).

Sufficient theorem conditions include identifiability, an interior parameter, suitable differentiability and domination, a consistent chosen maximizer, nonsingular information and local uniform remainder control. These are not established merely by listing a derivative and a second derivative. Plug-in standard errors add another consistency argument when information is evaluated at θ̂ rather than the unknown θ₀.

Apply

Contrast the maximum M of iid Uniform(0,θ) observations. For fixed t≥0 and n>t, P(n(1−M/θ)>t)=(1−t/n)ⁿ, which tends to e^(−t). Thus the nondegenerate limit is exponential on an n scale, not centered normal on a √n scale. The parameter-dependent support invalidates the regular template. Derive the probability from the maximum CDF rather than memorizing the conclusion.

If the model is misspecified, the estimate may target the parameter maximizing expected log likelihood, not a true generating parameter. Under separate conditions its covariance uses A⁻¹BA⁻¹, where A is the negative expected Hessian and B the score variance. The regular equality A=B can fail. A more sophisticated standard error does not repair an incorrectly interpreted target.

Continue

Label every reported normal approximation as asymptotic unless an exact law has been proved. Sample size alone is not a theorem checking regularity or approximation quality. Continue to The delta method for transforming a valid limit distribution. Boundary examples should stay nearby: they explain why a derivative-based approximation is a conditional mathematical result rather than an automatic property of estimators.