Track covariance when adding uncertain quantities

Try

Two quantities have means 4 and 2, variances 9 and 4, and covariance −3. Their sum has mean 6. Predict its variance before using a formula: negative association can make their deviations partly offset, reducing variability relative to an independent sum.

Predict

Compare adding X+Y with subtracting X−Y. A covariance term that reduces variability for the sum increases it for the difference. Predict why multiplying a variable by two quadruples its variance rather than doubling it, even when the variable’s mean simply doubles.

Experiment

The sum variance is 9+4+2(−3)=7. The difference variance is 9+4−2(−3)=19. For 2X+Y, variance is 4·9+4+4(−3)=28.

Combination	Mean	Variance
X+Y	6	7
X−Y	2	19
2X+Y	10	28

Check feasibility of the stated covariance: its magnitude 3 is below the SD product 3·2=6. A set of moment claims can be inconsistent, so a variance calculation should not blindly accept arbitrary covariance inputs.

Understand

For integrable variables, linearity gives E(aX + bY) = aE(X) + bE(Y). No independence assumption is required. For finite second moments, expand the centered square to obtain Var(aX + bY) = a²Var(X) + b²Var(Y) + 2abCov(X, Y). The cross term records dependence relevant to spread.

For a finite sum, variance equals the sum of individual variances plus twice the sum of pairwise covariances. Independence is sufficient to remove all covariance terms, but pairwise uncorrelatedness also suffices for that variance identity. It does not imply independence of the full collection or determine the sum’s complete distribution.

For iid observations with variance σ², the sample mean has variance σ²/n. This reduction depends on the cross terms being zero. If every observation is a repeated copy of the same random value, the average equals that value and its variance does not shrink at all. Counting repeated records as independent data would create fictitious precision.

Perfect dependence provides useful checks. If Y=X and Var(X)=v, then Var(X+Y)=4v and Var(X−Y)=0. Adding variances as if independent would produce 2v for both, revealing the missing covariance. If Y=−X, the sum instead has zero variance. These examples connect algebra to cancellation of actual random quantities.

In vector notation, the variance of a linear combination aᵀX is aᵀΣa, where Σ is the covariance matrix. Such a matrix must be positive semidefinite because every combination has nonnegative variance. This is a bridge to linear algebra and multivariate statistics, not permission to use arbitrary symmetric matrices as covariance models.

Lower variance is not automatically a better decision without a target and loss function. A combined estimate can be precise but biased, and a low-risk payoff can have an undesirable mean. Keep mean, uncertainty and the decision objective distinct.

Independent indicators need not have the same probability

Partition independent Bernoulli trials of common success probability p into disjoint blocks of lengths1,2,…,m. Let Iᵢ be one when every trial in block i succeeds, and zero otherwise. Define X as the number of successful blocks, the sum of these indicators. At p=1/2 and m=3, their means are1/2,1/4,1/8, so E[X]=7/8. A binomial model with three identical success probabilities does not describe this construction.

Same indicator means, different count distribution

Indicator i tests whether every Bernoulli trial in its length-i block succeeds. Predict how reusing nested prefixes changes the count law while keeping each indicator's probability p^i.

Blocks m 1234 Trial success p 0/41/42/43/44/4 Protocol Disjoint blocksNested prefixes Reset

• Indicator 1: trials 1; mean (2/4)^1.
• Indicator 2: trials 2, 3; mean (2/4)^2.
• Indicator 3: trials 4, 5, 6; mean (2/4)^3.

Uses 6 independent base trials; exactly enumerates 64 binary strings with their probability weights. Equal string weights apply only at p=1/2.

Count mean: 3584/4096 = 0.875. Variance: 9175040/16777216 = 0.546875.

Successful blocks	Probability
0	1344/4096
1	1984/4096
2	704/4096
3	64/4096

This is exact bounded enumeration, not a random simulation or empirical dataset. Different protocols use different numbers of base trials. At most10 trials and1024 strings; temporary settings do not establish independent learning or mastery.

For any specified vector of block indicator outcomes, sum the product probabilities of all underlying trial strings producing it. Because the blocks are disjoint and the base trials mutually independent, the allowed strings form a Cartesian product across blocks, and the finite sum factors into the product of the block-event probabilities. Thus the indicators are mutually independent, not merely uncorrelated. Each is Bernoulli with success probability pⁱ. Linearity gives E[X]=Σ pⁱ without requiring independence; independence removes all covariance terms and gives Var(X)=Σ(pⁱ−p²ⁱ), for i=1 through m.

For0 < p < 1 define Gₘ(t)=t(1−tᵐ)/(1−t). The mean is Gₘ(p), and variance is Gₘ(p)−Gₘ(p²). Handle endpoints from the mechanism: at p=0 every block fails and X=0; at p=1 every block succeeds and X=m; both variances are zero. Do not substitute p=1 into a displayed0/0 quotient. The complete count distribution is encoded by the polynomial product of ((1−pⁱ)+pⁱz): multiplying chooses success or failure for each independent block, and the coefficient of zʲ is P(X=j). This is a nonidentical Bernoulli sum, often called a Poisson-binomial distribution, not a Poisson distribution.

Reusing trials keeps the means but changes the law

Change the experiment: indicator Iᵢ now tests the first i trials of one length-m sequence. Each marginal remains pⁱ, but for i < j a success of Iⱼ forces Iᵢ=1. Thus E[IᵢIⱼ]=pʲ and Cov(Iᵢ,Iⱼ)=pʲ−pⁱ⁺ʲ. This is positive for0 < p < 1, so adding only the individual variances underestimates the count variance. The correct variance adds twice the sum of these covariances over i < j. For m=3,p=1/2, the disjoint variance is35/64 while the nested variance is71/64, although both means remain7/8.

The nested count equals the number of initial consecutive successes before the first failure, capped at m. Therefore P(X=j)=pʲ(1−p) for0 ≤ j < m and P(X=m)=pᵐ. The final atom includes the all-success sequence; it is not an uncapped geometric tail discarded and renormalized. This direct law independently checks the covariance calculation. Both protocols use independent base trials, but only disjoint blocks produce independent derived indicators. Their base-trial budgets also differ: m(m+1)/2 versus m. Equal marginal indicator probabilities do not make them the same experiment.

Apply

Derive the two-variable formula from centered variables, keeping the coefficient 2 in the cross term. Then use the repeated-copy example to explain why adding more dependent records may provide little new information. For the practice signed combination, track signs before substituting numbers.

Continue

Next condition before averaging. Conditional expectation makes the information available to a predictor explicit and leads to the tower and total-variance identities.