Connect sample moments to population parameters

Try

Imagine iid measurements uniformly distributed between zero and an unknown upper limit θ>0. Their population mean is θ/2. If the sample mean is 4, matching the sample and population first moments suggests θ̂=8. This is the method of moments: express population moments as functions of the parameter, replace them by sample moments, and solve. The procedure is motivated by laws of large numbers, but solving an equation does not automatically establish uniqueness, a permissible estimate, or good finite-sample behavior.

Predict

Predict whether an unbiased estimator must improve as more observations become available. For iid N(μ,1) data, the rule Tₙ=X₁ stays unbiased for every n but ignores all later observations. Its probability of missing μ by more than one remains a fixed positive number. In contrast, X̄+1/n is biased at every finite n but converges to μ. Write which property concerns expectations and which concerns probabilities of nonvanishing errors before looking at any algebra.

Experiment

Assume iid observations with variance 4. Chebyshev gives a simple, possibly loose, guarantee for the sample mean. For tolerance ε=1, the upper bound on error probability is 4/n, capped at one when necessary.

Sample size n	Var(X̄)	Chebyshev bound for error at least 1
4	1	1
16	1/4	1/4
64	1/16	1/16

Now halve the tolerance. The bound becomes sixteen divided by n, so maintaining the same bound requires four times as many observations. This is a consequence of the theorem’s squared tolerance, not an exact prediction of the actual tail probability. Identify both the guarantee and its possible conservatism.

Understand

Consistency of Tₙ for θ means Pθ(∣Tₙ−θ∣>ε) tends to zero for every fixed ε>0. For a finite-variance iid mean, Var(X̄)=σ²/n and Chebyshev proves consistency directly. An iid law of large numbers under a finite first absolute moment also gives consistency of the mean without requiring finite variance. Which theorem is available depends on the assumptions actually stated.

Moment estimates often use continuous mapping: if a sample moment converges to m(θ), and an inverse map g is continuous there, then g of the sample moment converges to θ. An inverse must be identifiable and defined on relevant inputs. For E X=θ² with θ≥0, the repaired estimate √max(X̄,0) is consistent because this map is continuous even at zero. Its derivative is not finite there, so this consistency argument supplies no ordinary first-order delta approximation at the boundary.

Apply

For Uniform(0,θ), derive Var(2X̄)=θ²/(3n) from Var(X)=θ²/12. The method-of-moments estimate is unbiased and consistent. However, a particular sample can have a maximum larger than twice its mean, making the estimated uniform support exclude an observed value. Try the synthetic sample 1,1,10: its estimate is 8, but an observation is 10. Moment matching optimizes neither support compatibility nor likelihood automatically. This concrete failure motivates checking the entire model after constructing an estimator.

Also distinguish convergence of estimators from convergence of their expectations. Rare, sufficiently large values can prevent expectation convergence even when convergence in probability holds. Additional moment or uniform-integrability control is needed to exchange a limit and expectation; consistency alone is not permission to do so.

Continue

Your proof should name the convergence theorem, its assumptions and the continuity step. Numerical stabilization in a finite simulation is evidence about that simulation, not a proof of consistency. Continue to Maximum likelihood and boundaries, which uses the full sampling density instead of only selected moments. Compare the two methods by their mechanisms and conditions before comparing their computed answers.