Recognize a common likelihood architecture

Try

Write the Bernoulli probability pˣ(1−p)¹⁻ˣ as an exponential. For 0<p<1 it becomes exp[x log(p/(1−p))+log(1−p)]. Define η=log(p/(1−p)). Solving gives p=e^η/(1+e^η), and the density becomes exp[ηx−log(1+e^η)] on x=0,1. The transformation reveals a common architecture: a parameter times a data statistic, minus a normalizing function that depends only on the parameter. The endpoints p=0,1 are limiting distributions rather than finite values of η.

Predict

Predict why differentiating the normalizer might reveal the mean. The probability law must still sum or integrate to one as η changes. Differentiating that identity links the data statistic to the normalizer’s slope. Differentiating again connects curvature to variability. These identities require justified interchange between differentiation and integration; the compact algebra does not remove that analytical condition. In a finite support model the interchange is immediate, while continuous models need domination or equivalent regularity.

Experiment

Compare three canonical representations. Here h includes the stated support, and A is the log normalizer. Check each representation by exponentiating and recovering the familiar probability law.

Model	Statistic T(x)	Natural parameter η	A(η)
Bernoulli	x	log(p/(1−p))	log(1+e^η)
Poisson	x	log λ	e^η
Normal, variance 1	x	μ	η²/2

For Poisson, h(x)=1/x! on nonnegative integers. For the normal row, h(x)=(2π)⁻¹ᐟ²exp(−x²/2). Differentiate A twice and recover the mean and variance of the displayed statistic in each row. Notice that information depends on the chosen parameterization: η is not always the ordinary mean parameter.

Understand

A canonical exponential family has density exp[η·T(x)−A(η)]h(x). The natural parameter space contains those η for which the normalizing integral is finite. In a scalar regular family, differentiation of the normalization gives Eη[T−A′(η)]=0, hence A′=ET. Another differentiation gives A″=Var(T). With several natural parameters, the gradient is the mean vector and the Hessian is the covariance matrix. A singular covariance signals a linear redundancy among statistics, which must be addressed before claiming nonsingular information.

For independent observations, products add canonical statistics. Thus the sample sum of T is sufficient by factorization. A full regular family has a natural parameter space with suitable open interior of the relevant dimension; a curved family restricts η to a lower-dimensional set. The familiar normal family with both mean and variance unknown uses T=(x,x²) and η=(μ/σ²,−1/(2σ²)), whose second coordinate is negative.

Apply

Now impose σ²=μ² with μ>0. The pair of natural coordinates becomes(1/μ,−1/(2μ²)), lying on a curve. Although the density still has two displayed statistics, its natural parameters do not range over an open subset of the plane. The usual full-family completeness theorem therefore cannot be imported solely by recognizing an exponential expression. This does not itself prove incompleteness; it identifies a missing theorem hypothesis.

Repair another common mistake: if η is changed to a nonlinear parameter θ, differentiating A as though θ were η drops chain-rule terms. Always name the coordinate in which the canonical moment identity is being used. Derive the Bernoulli variance in η coordinates, then transform the score back to p as a check.

Continue

The architecture is useful because it exposes exact factorization and derivative identities, not because it guarantees every inference procedure works. Support, natural-space geometry and integrability still matter. Continue to Rao–Blackwell improvement, where sufficient statistics improve estimators by conditioning. The following completeness lesson proves a finite-family case directly rather than relying on an unproved broad exponential-family theorem.