Categorical fit and model revision

Try

A synthetic inspection records the number of flagged positions in each of 100 two-position items. The observed counts for 0, 1 and 2 flags are 40, 40 and 20. Treat different items as independent observations with a common distribution. Before calculating, distinguish two questions: does a specified Binomial(2, 0.5) distribution fit, or does some Binomial(2, θ) distribution fit? The second question estimates θ from these same observations.

Each contribution is (observed−expected)²/expected. Recompute the middle row before adding the columns. A small expected count amplifies a fixed absolute discrepancy. The fitted distribution does not reproduce all three observed frequencies: its deviations are 4, −8 and 4.

Predict

Predict whether the two statistics should use the same reference distribution. Counts already have one constraint: their sum is 100. Fitting θ removes another direction of discrepancy, but we must prove which direction and why this estimator removes it.

The upper-tail asymptotic probabilities are approximately 0.002479 for 12 with two degrees of freedom, and 0.09558 for 25/9 with one degree of freedom. At a prechosen 5% level the specified distribution is rejected; the binomial family is not rejected. These answer different null hypotheses. Neither result proves that flags within an item are independent, that future items follow the same law, or that the fitted family is true.

Write the two null hypotheses in words. Then identify which assumptions come from the collection protocol rather than from this table. Choosing a new model because its probability is larger is exploratory model revision; that larger probability is not independent confirmation.

Experiment

Work on paper with a changed sample: 80 independent items produce counts (24, 32, 24). First fit θ by dividing the total number of flags by 160. Record the expected counts, the three contributions and their sum. Keep the category definitions fixed.

Check the changed calculation

There are 80 flags, so θ̂=1/2. Expected counts are (20, 40, 20), giving contributions 0.8, 1.6 and 0.8. Thus X²=3.2. Under the interior fitted-binomial calibration, the approximate upper-tail probability is 0.07364. This is a failure to reject at 5%, not acceptance of the model as established fact.

Now change the protocol rather than the numbers: suppose every inspected item was entered twice. The apparent sample size doubles and the Pearson statistic doubles, but no independent information was added. Explain why the multinomial reference calculation is then invalid. Finally, try merging categories 0 and 2 only after noticing their deviations. Such a data-dependent decision needs its own calibration; it cannot silently inherit the original test’s reference law.

Understand

A specified categorical distribution

Let n independent, identically distributed observations take values in k fixed categories, with specified probabilities p₁,…,pₖ, each positive and summing to 1. Let Oᵢ be the counts and p̂ᵢ=Oᵢ/n. Pearson’s statistic is X²=Σᵢ (Oᵢ−npᵢ)²/(npᵢ).

Here is the precise theorem dependency: the multivariate central limit theorem applied to the category-indicator vector gives √n(p̂−p) converging in distribution to a centered Gaussian vector with covariance diag(p)−ppᵀ. Its off-diagonal entries are −pᵢpⱼ because one observation cannot belong to two different categories. This lesson uses that central limit theorem, not a finite-sample normal identity.

Put s=(√p₁,…,√pₖ) and Z=diag(p)⁻¹ᐟ²√n(p̂−p). Its limiting covariance is P=I−ssᵀ. Since sᵀs=1, P is the orthogonal projection onto the (k−1)-dimensional space perpendicular to s. Indeed sᵀZ=0 exactly because the frequencies sum to 1. In an orthonormal basis of that space the limiting Gaussian has k−1 independent standard normal coordinates. By continuity of squared norm, X²=‖Z‖² converges in distribution to χ² with k−1 degrees of freedom.

This is an asymptotic conclusion for fixed positive p and fixed k as n grows. A rule such as expected counts at least 5 is a screening convention, not the proof or a guarantee of a particular approximation error. Sparse cells may require a finite multinomial calculation or simulation of the specified collection protocol. Deliberately combining categories in advance changes the question and the number of support cells.

If pᵢ=0 and Oᵢ>0, the specified null assigns probability zero to the observation. Report that contradiction; do not divide by zero and return an ordinary chi-square probability. If pᵢ=Oᵢ=0, omit that structural-zero category from the support before computing the statistic and its degrees of freedom. One positive-probability category is deterministic, not an ordinary positive-degree chi-square test. Dependence, unequal item distributions or appreciable sampling without replacement require a different covariance analysis.

Fitting the three-cell binomial family

For θ in (0,1), p(θ)=((1−θ)², 2θ(1−θ), θ²). The likelihood, apart from a factor independent of θ, is θˢ(1−θ)^(2n−S), where S=O₁+2O₂. Differentiating its log gives S/θ−(2n−S)/(1−θ). When 0<S<2n, strict concavity gives the unique maximum θ̂=S/(2n). If S=0 or S=2n the maximum is at 0 or 1. The law of large numbers gives θ̂→θ under the model, since each observation has mean 2θ.

We now justify one degree of freedom rather than invoking a parameter-count slogan. Fix the true interior θ, and let s and Z use p(θ). Define vᵢ=pᵢ′(θ)/√pᵢ(θ). Then

v=(−2, √2(1−2θ)/√(θ(1−θ)), 2), sᵀv=0, Iθ=vᵀv=2/[θ(1−θ)].

The score identity pᵢ′/pᵢ=(i−2θ)/[θ(1−θ)] gives the exact relation √n(θ̂−θ)=Iθ⁻¹vᵀZ. The subtraction of 2θ vanishes because Σ√pᵢ Zᵢ=0. Thus fitting removes precisely the component of Z along v.

More explicitly, Taylor expansion of the quadratic vector p(θ̂), together with √n(θ̂−θ) bounded in probability, shows that the standardized fitted residual vector equals (I−vvᵀ/Iθ)Z plus a vector converging to zero in probability. Replacing √pᵢ(θ) by √pᵢ(θ̂) in the denominators has the same vanishing effect because the true probabilities are positive. The limiting covariance is therefore I−ssᵀ−vvᵀ/Iθ, an orthogonal projection of rank 1. Its squared norm has the χ²₁ law. This uses Taylor expansion, the multivariate CLT and the usual continuous-mapping/Slutsky rules.

For a concrete check, the surviving unit direction is w=(θ, −√(2θ(1−θ)), 1−θ): w is perpendicular to both s and v. For any interior fitted sample put δ=p̂₂−θ̂². Its residual probability vector is exactly (δ,−2δ,δ), and direct substitution gives X²=nδ²/[θ̂²(1−θ̂)²]. The fit tests a remaining shape constraint, not simply whether the sample mean is close to its fitted mean.

At θ=0 or 1 the positive-cell and tangent assumptions fail. The model is deterministic; the interior χ²₁ proof does not apply. Even an interior model can produce a finite sample whose MLE is on the boundary. Report that regular calibration is unavailable for that sample. Estimating a parameter by an arbitrary rule, selecting a family after inspection, or having redundant parameters does not justify subtracting the number of displayed parameters.

Apply

A proposed report says: “All distributions have three cells, so every test has two degrees of freedom. We tried several families, kept the one with the largest probability, and confirmed it on the same counts.” Repair each sentence. State the design assumptions, the exact null, whether its parameters were specified or fitted, the statistic, the reference justification and the uncertainty in the conclusion.

Compare a defensible revision

Three fixed positive cells give two degrees of freedom under iid sampling. This particular interior binomial MLE gives one, by the tangent projection proved above. Searching among families makes the selected result exploratory. A useful next step is to specify the candidate family and a fresh validation protocol before collecting more data. One may refit this prechosen family on the independent validation counts and use its justified fitted reference. Alternatively calibrate an explicitly specified whole selection procedure. Merely treating a noisy training estimate as the unknown true parameter on new data does not remove its estimation uncertainty.

For a separate structural-zero example, use p=(0,0.4,0.6) and counts (0,18,32). The support has two categories, so the statistic is 4/20+4/30=1/3 and its regular asymptotic reference has one degree of freedom. Changing the first count to a positive number contradicts that null outright. A zero observed count in a positive-probability category is different from a structural zero.

Continue

Return to statistical models to state the observation law, and maximum likelihood and boundaries to separate interior fitting from boundary optimization. The asymptotic likelihood lesson supplies broader estimation context; the projection proof here establishes the specific categorical calibration used in this investigation.

Keep the original count table, your prechosen category definitions and the revised collection plan together. A test result is evidence about a stated probability model under a stated observation protocol, not a certificate that every mechanism behind the data has been identified.