40 practice points.
Question 9: A fitted line and two kinds of uncertainty
10 points
Fit y=β₀+β₁x+ε to the four synthetic rows below. For conditional inference assume fixed x, independent Normal(0,σ²) errors and a correct linear mean. Use unbiased residual variance SSE/(n−2). For a design row v=(1,x₀), mean variance is σ²v(XᵀX)⁻¹vᵀ and future-observation variance adds σ². The 0.975 quantile of t₂ is 4.303.
(a) · 5 points
Compute XᵀX, fitted coefficients, all residuals, SSE, residual variance and the estimated coefficient covariance. Verify both normal-equation residual conditions.
(b) · 5 points
At x₀=1 compute the fitted mean and 95% mean-response and prediction intervals. Explain why zero residual sum neither verifies linearity nor establishes causal effects, and what changes under heteroscedastic errors.
Review question 9: worked argument and scoring criteria
(a)
XᵀX=[[4,2],[2,6]] with inverse [[0.3,−0.1],[−0.1,0.2]]. Xᵀy=(6,9), so β̂=(0.9,1.2). Fitted responses are (−0.3,0.9,2.1,3.3), residuals (0.3,0.1,−1.1,0.7), whose sum and x-weighted sum are zero. SSE=0.09+0.01+1.21+0.49=1.8, s²=0.9, and estimated coefficient covariance is [[0.27,−0.09],[−0.09,0.18]].
- 1 point: Computes the design cross-product and inverse.
- 1 point: Computes coefficients and residuals.
- 1 point: Checks both residual orthogonality conditions.
- 1 point: Computes SSE and residual variance.
- 1 point: Computes the full covariance matrix.
(b)
The fitted mean is 2.1. v(XᵀX)⁻¹vᵀ=0.3, so estimated mean variance is 0.27 and future-response variance is 1.17. The intervals are 2.1±4.303√0.27≈[−0.136,4.336] and 2.1±4.303√1.17≈[−2.554,6.754]. Residual orthogonality follows mechanically from least squares with an intercept and cannot verify the mean specification; observational association also does not establish intervention effects. Under unequal variances the covariance is (XᵀX)⁻¹XᵀΩX(XᵀX)⁻¹, not σ²(XᵀX)⁻¹, and the stated exact t reference is unjustified.
- 2 points: Computes the two variances and intervals.
- 1 point: Explains mechanical residual orthogonality.
- 1 point: Distinguishes association from causation.
- 1 point: Identifies the heteroscedastic covariance and calibration change.
Targeted lesson returns:
Question 10: Coordinates are part of the analysis
10 points
Four synthetic two-variable observations are (−2,0),(0,−1),(0,1),(2,0). Use centered sample covariance with divisor n−1. Principal directions have unit Euclidean length. The sign of a direction is arbitrary but must agree with the scores and reconstruction.
(a) · 5 points
Compute the mean, covariance, eigenvalues and a leading loading. Give all first-component scores, rank-one reconstructed rows, explained-variance fraction and total squared reconstruction error.
(b) · 5 points
Multiply the second measured coordinate by 3 and redo the leading-direction and explained-variance calculation. What happens if each coordinate is standardized by its sample standard deviation first? Explain what neither analysis proves about a population.
Review question 10: worked argument and scoring criteria
(a)
The mean is (0,0). The covariance is diag(8/3,2/3). Eigenvalues are 8/3 and 2/3, with leading loading (1,0) (or its negative). Scores are (−2,0,0,2), reconstructing (−2,0),(0,0),(0,0),(2,0). Explained variance is (8/3)/(10/3)=4/5. The squared reconstruction error is 0+1+1+0=2=(n−1) times the discarded eigenvalue.
- 1 point: Computes centering and covariance with the correct divisor.
- 1 point: Obtains eigenvalues and a unit loading.
- 1 point: Gives consistent scores and reconstructions.
- 1 point: Computes 4/5 explained variance.
- 1 point: Computes and checks the reconstruction error.
(b)
After rescaling, covariance is diag(8/3,6), so the leading direction becomes (0,1), with explained fraction 6/(6+8/3)=9/13. Standardizing either version yields the identity correlation matrix, whose eigenvalues are both 1. Every unit direction then maximizes sample projected variance; there is no unique first component, and any one component explains half the standardized variance. Unit changes affect covariance PCA. None of these sample calculations proves a population latent factor, a causal mechanism or stable loadings on future data; external validation and uncertainty assumptions are separate.
- 2 points: Computes the rescaled leading direction and 9/13 fraction.
- 2 points: Explains the standardized tie and nonuniqueness.
- 1 point: States a concrete statistical interpretation limit.
Targeted lesson returns:
Question 11: A fitting step and a nonfinite optimum
10 points
For grouped binomial observations, yᵢ successes out of mᵢ trials are conditionally independent with logit pᵢ=xᵢᵀβ, where logit p=ln[p/(1−p)]. Ignoring binomial constants, ℓ(β)=Σ[yᵢxᵢᵀβ−mᵢln(1+exp(xᵢᵀβ))]. First consider an intercept-only model with two rows (m,y)=(4,1),(6,3).
(a) · 5 points
Derive the vector score and negative Hessian from the supplied likelihood. For the intercept-only data compute one Newton step from β=0 and the finite MLE. State the conditional success-probability estimate.
(b) · 5 points
Now use two Bernoulli rows with intercept and slope: (x,y)=(−1,0),(1,1). Prove there is no finite maximum along intercept 0 and increasing slope, despite full design rank. Contrast this with an intercept-only Poisson log-link model having independent counts 0,1,2 and unit exposures; derive its MLE and estimated inverse information.
Review question 11: worked argument and scoring criteria
(a)
Differentiation gives U=Xᵀ(y−mp) and I=XᵀWX with Wᵢᵢ=mᵢpᵢ(1−pᵢ). The Newton ascent step is βnew=β+I⁻¹U when I is invertible. At β=0 all pᵢ=1/2, U=4−10/2=−1 and I=10/4=2.5, so the first step is −0.4. The intercept score vanishes at p=4/10=2/5, hence β̂=logit(2/5)=ln(2/3)≈−0.405465. The first iterate is close to, but not equal to, the maximum. With successes and failures both present, the strictly concave scalar likelihood has this finite maximum.
- 2 points: Derives the vector score and curvature.
- 1 point: States the correctly signed Newton update.
- 1 point: Computes the step and fitted probability.
- 1 point: Justifies existence and uniqueness in this case.
(b)
At intercept 0 and slope b, both observed outcomes have fitted probability sigmoid(b), so ℓ(b)=2ln sigmoid(b) increases toward 0 as b→∞, never attaining it at finite b. Each Bernoulli log probability is at most 0, so this is the global supremum and no finite maximizer exists. The two design rows (1,−1),(1,1) have full rank: rank does not preclude separation. For the Poisson model ℓ(β)=3β−3e^β plus constants; score 3−3e^β gives β̂=0 and fitted mean 1. Negative Hessian at the optimum is 3, so inverse information is 1/3. That last quantity is an asymptotic variance estimate, not a justified exact normal interval for three counts.
- 2 points: Proves separation and nonattainment of the likelihood supremum.
- 1 point: Separates rank from finite-MLE existence.
- 1 point: Derives the Poisson fit and inverse information.
- 1 point: Labels the uncertainty approximation accurately.
Targeted lesson returns:
Question 12: A frozen comparison with a changed target
10 points
Two models were fitted and frozen before observing the four synthetic validation responses below. The intended loss is mean squared error in the original response units. The validation cases are drawn independently from the intended population for this exercise. No extra test set has yet been collected.
(a) · 5 points
Compute both mean squared errors and the four paired differences (squared error of A minus that of B). Compute their sample mean and sample standard deviation. What does pairing preserve, and what population superiority claim is justified without further assumptions?
(b) · 5 points
The team now selects B and revises its predictions after inspecting these errors. Explain what must change before claiming fresh performance evidence. If the target changes from y to 1/y for positive y, prove why reciprocating a conditional-mean predictor generally fails to produce the new conditional mean, using Y=1 or 3 with equal probabilities as a counterexample.
Review question 12: worked argument and scoring criteria
(a)
A has squared errors (1,0,1,4), MSE 6/4=1.5. B has (0,1,1,0), MSE 2/4=0.5. Paired differences are (1,−1,0,4), mean 1. Their squared deviations from 1 are (0,4,1,9), so sample variance is 14/3 and sample standard deviation √(14/3)≈2.160. Pairing compares both models on the same cases and retains shared case difficulty. B has lower observed loss by 1; four cases and these calculations alone do not prove population superiority. An exact t claim would require a suitable normal difference model, not merely independence of cases.
- 2 points: Computes both error vectors and MSEs.
- 1 point: Computes paired differences and their mean.
- 1 point: Computes the sample spread with divisor 3.
- 1 point: Explains pairing and the inferential limit.
(b)
The four responses have become development evidence for selection and revision; relabeling or resetting them cannot make them fresh. Freeze the entire revised pipeline and evaluate on new independent target-population cases, recording prior exposure, the selected loss and uncertainty. For the reciprocal target, E(1/Y|x) is generally not 1/E(Y|x). In the equally likely {1,3} example, E(Y)=2 so 1/E(Y)=1/2, whereas E(1/Y)=(1+1/3)/2=2/3. Squared-error optimality for each target requires its own conditional expectation; changing the target can therefore change the optimal model and assessment.
- 2 points: Preserves exposure and specifies a fresh frozen-pipeline evaluation.
- 2 points: Derives the reciprocal counterexample.
- 1 point: Connects target changes to conditional-mean optimality and loss.
Targeted lesson returns: