Fit a line and inspect its failures

Try

For pairs (0,1), (1,2), (2,5), try a line with slope two. No single intercept makes it pass through all three points. Predict how a fitting rule could compromise between overprediction and underprediction. Simply minimizing signed errors is unsuitable because positive and negative errors cancel.

Predict

Predict whether the fitted line must pass through every observation or through the point of sample means. Also predict what a curved pattern in residuals would mean. A small average signed error does not establish that the model has captured the relationship.

Experiment

For these synthetic data, x̄=1 and ȳ=8/3. The squared-x-deviation sum Sxx is 1+0+1=2. The cross-deviation sum Sxy is 5/3+0+7/3=4. Least-squares slope b=Sxy/Sxx=2, and intercept a=ȳ−bx̄=2/3.

x	Observed y	Predicted y	Residual
0	1	2/3	1/3
1	2	8/3	−2/3
2	5	14/3	1/3

Check that residuals sum to zero and their squared sum is 2/3. In the live regression investigation, change the final observation and inspect how every fitted value can move.

Understand

Least squares chooses coefficients minimizing the sum of squared residuals for the stated data and model. The fitted value is ŷ=a+bx, where x is the predictor, b is slope and a is intercept. A residual is observed minus predicted: e=y−ŷ. It has the response variable’s units. A negative residual means the model predicted too high.

The slope formula requires variation in x: Sxx must be positive. With all x values identical, the usual slope is not identified. For an unconstrained least-squares line with an intercept, the line passes through (x̄,ȳ), and residuals sum to zero. These algebraic properties do not guarantee that a straight line is an appropriate scientific model.

Plot residuals against x or fitted values. A curved pattern suggests the mean relationship may not be linear. A widening spread suggests variability changes with x. Clusters or time patterns can indicate omitted structure or dependence. These are reasons to investigate, not automatic instructions to transform or delete observations. Inference about a slope requires additional assumptions and methods beyond fitting a line.

In simple linear regression with an intercept and nonzero response variation, r² equals the fraction of the observed total squared response variation accounted for by the fitted line. It does not measure the fraction of outcomes caused by x, nor guarantee accurate prediction for new cases. High fit can coexist with biased sampling or an inappropriate causal story.

Interpolation predicts inside the observed x range; extrapolation goes outside it. A battery model fitted from 10–30°C may be unreliable at 80°C because the underlying mechanism can change. Even interpolation needs care when relevant subgroups or operating conditions differ from the training data.

Apply

Fit the small example by hand and check the residual direction against the graph. In the live investigation, compare including and excluding the unusual point, then write what independent evidence would justify exclusion. Report how the slope changes without calling the preferred slope the true effect.

Continue

Next examine transformations. A new scale can clarify structure, but it changes how coefficients and averages must be interpreted.