Measure local likelihood sensitivity

Try

For one Bernoulli observation X with 0<p<1, differentiate its log likelihood with respect to p. The score is U=X/p−(1−X)/(1−p)=(X−p)/(p(1−p)). It is positive after a success and negative after a failure. Its mean is zero, and its variance is 1/[p(1−p)]. This variance is Fisher information for the coordinate p. It quantifies local likelihood sensitivity under the model; it is not the amount of factual certainty supplied by one observed outcome.

Predict

Predict what happens with n independent observations. Their scores add. Because each score has mean zero and different scores are independent, cross terms vanish when computing the variance. Information becomes n/[p(1−p)]. The reciprocal equals p(1−p)/n, the variance of the sample proportion. This suggests an optimality bound. Notice that information diverges near a boundary in this coordinate, but the regular theorem we will derive is an interior statement; plugging in an endpoint is not justified by taking a formula literally.

Experiment

Check the score distribution at p=1/3 using exact fractions. Then compute its mean and second moment from the two rows.

Outcome	Probability	Score U
X=0	2/3	−3/2
X=1	1/3	3

The mean is −1+1=0. The second moment is (2/3)(9/4)+(1/3)9=9/2. For two observations information is 9, so the unbiased variance bound for p is 1/9. Derive Var(X̄) separately and confirm equality. The agreement is a theorem check for this regular model, not a universal property of maximum likelihood estimators.

Understand

Let fθ have a common parameter-independent support, differentiable density and justified differentiation under the needed integrals. Write U=∂log fθ/∂θ and assume finite positive information I(θ)=EθU². Differentiating ∫fθ=1 gives EU=0. If T is unbiased for a differentiable target g(θ), differentiating ET=g gives E(TU)=g′. Hence Cov(T,U)=g′. Cauchy–Schwarz yields [g′]²≤Var(T)I, or Var(T)≥[g′]²/I. The derivative interchange for the T-weighted integral is a separate condition from merely differentiating the density pointwise.

Equality in Cauchy–Schwarz requires T−g(θ) to be a scalar multiple of U almost surely at the parameter being considered. The scalar is g′/I when the identities hold. Achieving equality at every parameter can place strong restrictions on the family and estimator. The alternative identity I=−E of the log-likelihood second derivative requires further justified differentiation; it should not be used as a definition that bypasses regularity.

Apply

For Uniform(0,θ), the interior log-density derivative is −1/θ, whose mean is not zero. The support moves as θ changes. Differentiating the normalization must include a boundary contribution, so the regular score proof breaks at its first interchange. There is no contradiction with probability theory. The correct conclusion is that this theorem’s assumptions are absent, not that uniform data violate a universal variance law.

For a normal mean with known variance σ², the sample score is n(X̄−μ)/σ². The centered sample mean is exactly σ²/n times that score, so it attains the bound σ²/n. Identify both the target coordinate and known-variance assumption before transferring the calculation to a new problem.

Continue

Information depends on parameterization: replacing θ by a smooth coordinate introduces derivative factors. Its role in a variance bound transforms accordingly. Keep the score, its expectation, support and differentiation conditions visible. Continue to MLE asymptotics to see why inverse information also appears in a regular large-sample approximation. Exact finite-sample attainment and asymptotic efficiency are different claims and need different arguments.