Find the assumptions that drive a reported standard error

Try

In an intercept-only model, every fitted value is the sample mean and every residual is an observation minus that mean. The residuals sum to zero, so they cannot be independent copies of the original errors. Even if errors were independent normal variables, residual construction introduces dependence. A residual plot is therefore a view of transformed data, not a direct observation of independent noise. Start by deriving its covariance before treating its features as evidence about the error model.

Predict

Predict which observations have smaller raw residual variance under a correct homoskedastic model. A high-leverage point can pull the fit toward itself, reducing its own residual variance. Thus a small raw residual does not guarantee low influence or an ordinary predictor location. Leverage depends only on the design matrix, while influence also depends on the observed response. These concepts should not be collapsed into one label for an observation that looks unusual.

Experiment

In the intercept-only model with n=4, H has every entry equal to 1/4. Assume original errors have variance σ²=4 and are uncorrelated. Compute the entries of Cov(e)=σ²(I−H).

Quantity	Calculation	Value
Each leverage	hᵢᵢ	1/4
Each residual variance	4(1−1/4)	3
Covariance of distinct residuals	−4/4	−1

Check that the variance of the sum of all four residuals is zero: four diagonal terms contribute 12 and twelve ordered cross terms contribute −12. This calculation connects the covariance matrix with the exact sum-to-zero constraint. It also shows why using an iid-error interpretation for the residual vector would be mathematically inconsistent.

Understand

For full-rank fixed design, e=(I−H)ε. If Cov(ε)=σ²I, symmetry and idempotence give Cov(e)=σ²(I−H). The diagonal entries hᵢᵢ lie between zero and one and sum to p, the rank of H. To see the bounds, hᵢᵢ is the squared length of the projection of a coordinate unit vector, which cannot exceed its original squared length. Standardizing residuals by their model-based scales can account for leverage, but estimated scales and model selection introduce further dependence.

If Cov(ε)=Ω instead, ordinary least squares has covariance (XᵀX)⁻¹XᵀΩX(XᵀX)⁻¹. Heteroskedasticity-robust or dependence-robust estimators attempt to estimate appropriate versions of this sandwich under additional asymptotic assumptions. Writing the population formula does not prove that a particular residual-based estimate is accurate in a small sample. The mean model, dependence structure, leverage and available replication still matter.

Apply

Prove the Gauss–Markov comparison under Eε=0 and Cov(ε)=σ²I. Any linear unbiased estimator AY satisfies AX=I. Write A=AOLS+D, so DX=0. The cross covariance terms vanish, leaving Cov(AY)−Cov(β̂)=σ²DDᵀ, a positive-semidefinite matrix. Thus OLS is best in the class of linear unbiased estimators under those moment assumptions. This is not a claim about every nonlinear or biased estimator, and normality is unnecessary for this particular proof.

Suppose a residual plot shows spread increasing with fitted values. Consider heteroskedasticity, a wrong mean function, omitted grouping and influential observations. Propose one targeted investigation for each plausible mechanism rather than immediately applying a single correction. If a transformation is selected after examining many plots, subsequent uncertainty statements should acknowledge that model-selection process. A diagnostic pattern suggests questions; a pattern’s absence does not prove the model assumptions.

Continue

Report the design, fitted mean structure, residual evidence, sensitivity checks and remaining uncertainties together. A covariance correction cannot repair selection bias or turn an associational coefficient into a causal effect. Continue to Resampling and limitations to examine another route to uncertainty assessment. Resampling is also a model-dependent procedure: the way observations are resampled asserts a structure about how new data would arise.