Compare constrained and unconstrained fits

Try

Fit a normal mean with known variance σ². Under the null μ=μ₀, the mean is fixed; under the full model it is estimated by X̄. The likelihood ratio Λ is the largest null likelihood divided by the largest full-model likelihood. Because the null parameter set is contained in the full set, Λ lies between zero and one when these quantities are well defined. Small Λ means the constraint costs substantial fit. It is not a probability that the constraint is true.

Predict

Predict the relation between the likelihood loss and the standardized distance between X̄ and μ₀. Use the identity Σ(xᵢ−μ₀)²=Σ(xᵢ−X̄)²+n(X̄−μ₀)². The difference is nonnegative and depends only on the mean discrepancy. Multiplying the log-likelihood difference by two gives W=−2 logΛ=n(X̄−μ₀)²/σ². Under the null this is the square of a standard normal, so its chi-square law is exact here, without a large-sample theorem.

Experiment

Let n=4, σ²=4 and μ₀=0. Compare the unrestricted and nonnegative-mean models for several observed means. The second model is used to test its boundary point μ=0.

Observed mean	Unrestricted W	W with μ constrained nonnegative
−2	4	0
0	0	0
2	4	4

Explain the zero in the first row of the last column. The constrained full-model maximizer is max(X̄,0), so negative means provide no improvement over the null. The shape of the parameter space has changed the sampling distribution of the likelihood-ratio statistic even though the observation model is still normal.

Understand

Wilks’ theorem supplies an asymptotic chi-square calibration in regular nested finite-dimensional models. A useful form requires an identifiable smooth model, an interior true parameter, nonsingular information, consistent fits and a smooth null submodel with well-defined codimension. Under appropriate local regularity, W converges to chi-square with degrees of freedom equal to the difference in dimensions. The mechanism is a quadratic approximation to log likelihood and a normal score projected onto directions excluded by the null. This is a proof sketch; the needed uniform stochastic remainder control is part of the theorem, not established by drawing a parabola.

Parameter counting alone is insufficient. A boundary, a parameter not identifiable under the null, a singular information matrix, or a growing-dimensional model can invalidate the usual reference distribution. Mixture-component tests are a common structural warning because a component’s parameters can disappear when its weight is zero. One cannot repair such failures by mechanically subtracting the two parameter counts.

Apply

For a single N(μ,1) observation with μ≥0, testing μ=0 gives W=(max(X,0))². Under the null, X is equally likely to be positive or negative. The statistic has mass one half at zero, and its positive part contributes half of a χ²₁ distribution. Derive this mixture using symmetry and the normal square. A pure χ²₁ calibration is therefore not the exact boundary law. This finite example makes the regularity failure explicit rather than merely warning that “small samples are difficult.”

Also compare likelihood ratios with posterior odds. The former maximize sampling likelihoods over parameter sets; the latter integrate with priors and include prior odds. They answer different questions even when they favor the same model. State the calculation and calibration used instead of relabeling one as the other.

Continue

Before using a chi-square reference, distinguish an exact derivation from a Wilks approximation and list the relevant regularity conditions. Continue to Confidence inversion and power, which turns valid tests into confidence sets. A confidence set inherits the calibration quality of the tests it inverts; inversion cannot transform an unjustified approximation into an exact guarantee.