Try
Suppose four individual Bernoulli observations have predictors x=−2,−1,1,2 and outcomes 0,0,1,1. The slope c in η=cx can increase without changing which side of probability 1/2 each row occupies. As c grows, every observed outcome becomes more probable. Will a finite best coefficient ever be reached?
| x | Observed outcome | Predictor at slope c | Probability as c increases without bound |
|---|
| −2 | 0 | −2c | tends to 0 |
| −1 | 0 | −c | tends to 0 |
| 1 | 1 | c | tends to 1 |
| 2 | 1 | 2c | tends to 1 |
Every finite coefficient gives probabilities strictly between 0 and 1. The sum of observed log probabilities is therefore strictly negative, although it approaches 0 along this path. Since no log probability exceeds 0, this is the supremum and it is unattained. The finite unpenalized maximum-likelihood estimate does not exist.
Predict
Now use x=−1,0,1 with totals (1,2,1) and successes (0,1,1). The middle row contains both outcomes. Does this single mixed row rule out an infinite slope?
Check the middle row's direction
Along β=(0,c), its predictor remains zero and its probability remains 1/2, maximizing that row’s likelihood. The two outer pure rows improve as c grows. Omitting binomial constants, the supremum is −2 log 2 and is unattained. This is quasi-separation: some constraints are equalities while at least one row improves strictly. A mixed row prevents escape only in directions that change its predictor.
Experiment
Use the GLM workbench. Compare Complete separation, Quasi separation and Collinear predictors. Initialize each before requesting a step or bounded run. Record separately the existence certificate, design rank and numerical status. Collinearity means some coefficient changes leave every predictor unchanged; separation means a nonzero direction improves likelihood without a finite optimum. They can occur together, but they are not the same defect.
Return to the worked binomial example. Its mixed rows span both coefficient coordinates, so the finite-existence result below applies. A numerical fit must still solve the problem to its stated tolerance. The current experiment caps a run at 80 iterations and guards linear predictors outside its supported ±40 range. A finite mathematical answer can lie beyond a tool’s arithmetic scope. A cap or numerical failure is not a proof that no mathematical answer exists.
Write the last accepted coefficient vector and why the run stopped. Cancellation or exhausted work must not convert the last candidate into a completed maximum-likelihood fit. If existence is reported as unknown, retain that word; do not rewrite it as either “finite” or “separated.” A found and verified separating direction is positive evidence of nonexistence, whereas failing to find one need not settle the problem.
Understand
A sufficient finite-existence certificate
Consider independent grouped-binomial rows, positive trial counts and a full-column-rank design. Suppose the rows with 0<sᵢ<nᵢ themselves span the coefficient space. For each such mixed row, the log-likelihood term sᵢη−nᵢ log(1+exp η) tends to −∞ at both ends of the real line: its slopes in the two tails have the required opposite signs.
Write Xₘ for the mixed-row submatrix. Full rank gives a constant c>0 with ||Xₘβ||≥c||β||. One proof minimizes the continuous positive function ||Xₘd|| on the compact unit sphere. Thus, as ||β|| grows without bound, at least one mixed-row predictor must grow in absolute value. There are finitely many rows, each mixed row penalizes both tails, and all the remaining binomial terms are bounded above by zero. Consequently the total log likelihood tends to −∞ as ||β||→∞: it is coercive downward.
Choose any finite starting coefficient vector. Its superlevel set is closed and bounded, hence compact. Continuity gives a maximizer there, and values outside are smaller than the starting value. Strict concavity from positive information then proves uniqueness. Fixed finite offsets do not change the tail argument.
For Poisson regression, a positive-count term yη−exp η tends to −∞ at both ends when y>0. Every zero-count term −exp η is bounded above by zero, and positive-count terms have finite upper bounds. If the positive-count rows span the coefficient space, the same argument proves finite existence and uniqueness. This is why the worked Poisson data admit that certificate.
These are sufficient conditions. They are deliberately not asserted to be necessary. For example, Bernoulli outcomes (1,0,1) at x=−1,0,1 have no mixed rows. Nevertheless β=(log 2,0) has zero score; strict concavity proves it is the unique finite maximizer. An engine’s conservative existence check can remain unknown when another mathematical proof succeeds.
A recession direction is a different certificate
For binomial rows, let d be a direction in coefficient space. A pure-success row permits aᵢᵀd≥0, a pure-failure row permits aᵢᵀd≤0, and a mixed row requires aᵢᵀd=0. If these hold and some pure row has strict inequality, moving along d strictly increases likelihood from every finite starting point. No finite maximizer can exist. The sign proof applies to any starting coefficients, not just a plotted line through zero.
For Poisson rows, the corresponding sufficient recession conditions are aᵢᵀd=0 at every positive-count row and aᵢᵀd≤0 at every zero-count row, with at least one strict inequality. Positive-row means stay fixed and at least one zero-row mean decreases, strictly increasing likelihood. An intercept-only all-zero dataset is the simplest case: β→−∞ approaches rate 0, which is a boundary rate rather than a finite log-rate coefficient.
The workbench verifies any reported witness within its declared arithmetic rules. A large coefficient, a near-zero weight or a stalled iteration alone is not this proof. Adding a penalty can create a different optimization problem with different interpretation; it must be an explicit model revision, not a hidden numerical repair.
Four questions to keep separate
- Does the design identify coefficients?
- Is a finite optimizer known to exist, known not to exist, or unresolved by the available certificate?
- Did the numerical algorithm meet its stopping rule within its supported work and arithmetic limits?
- Are the distributional and asymptotic conditions for an uncertainty statement credible?
A favorable answer to one question does not supply the others. Record all four in an analysis report.
Apply
An intercept-only Poisson model has positive exposures (1,2,3). First take counts (0,0,0); then change only the last count to 2. Derive both likelihood conclusions without running an optimizer. Does one zero row prevent a finite answer?
Check the boundary and changed total
With total exposure 6 and total count Y, the relevant log likelihood is Yβ−6 exp β. When Y=0, its supremum is approached as β→−∞ and no finite β attains it. When Y=2, the score vanishes at exp β=2/6=1/3, hence β=−log 3; the second derivative is negative and this is the unique maximum. Individual zero rows are compatible with a finite estimate. The all-zero boundary, not the mere presence of a zero, caused the first failure.
Continue
In GLM uncertainty and model checks, distinguish a finite numerical result from an asymptotic sampling approximation. Return to score and information to reconstruct the local calculation.
