Statistical inference: estimation and hypothesis testing

Before an election a polling firm asks 1000 people and reports “52% ± 3%”. A tea factory weighs 30 packets rather than each of the millions it produces. A teacher wonders whether a new method really raised the scores or the class was just lucky. Statistical inference answers such questions: how to draw reliable conclusions about a whole population from a random sample, and how sure we can be.

Population, sample and estimators

Why n − 1? The deviations are measured from x̄, which was fitted to the same data, so on average they come out slightly too small. The deviations always add up to 0, so only n − 1 of them are free (the degrees of freedom). Dividing by n − 1 instead of n corrects this bias exactly.

Confidence intervals for the mean

A single number x̄ never hits μ exactly. A confidence interval is a range built so that in 95% of samples it contains μ. By the central limit theorem x̄ ≈ N(μ, σ²/n), so with probability 0.95, μ lies within 1.96 standard errors of x̄.

Hypothesis testing

1. 1
   State the hypotheses

   The null hypothesis H₀ is the “nothing special” claim (μ = 500 g, the method has no effect). The alternative H₁ is what we suspect (μ < 500 g, the method helps).

2. 2
   Choose the significance level

   α is the risk of wrongly rejecting a true H₀ that we accept; usually α = 0.05 (sometimes 0.01).

3. 3
   Compute the test statistic

   It measures how far the data are from H₀ in units of the standard error, for example z or t.

4. 4
   Find the p-value

   The probability, assuming H₀ is true, of a result at least as extreme as the one observed.

5. 5
   Decide

   If p ≤ α, reject H₀ (the result is statistically significant). If p > α, do not reject H₀: the data are not strong enough evidence, which is not a proof that H₀ is true.

The t-test. When σ is unknown we replace it with s and compare t with Student's t distribution (df = n − 1). A two-sample t-test compares the means of two groups, for example classes taught with a new and an old method, by dividing the difference of the means by its standard error. The t distribution was published in 1908 by William Gosset, who worked at the Guinness brewery in Dublin and wrote under the pen name “Student”.

Correlation and linear regression

The regression line ŷ = b₀ + b₁x is chosen by the least squares method: it minimises the sum of squared vertical errors S(b₀, b₁) = ∑(yᵢ − b₀ − b₁xᵢ)². Setting the partial derivatives ∂S/∂b₀ and ∂S/∂b₁ equal to zero gives two linear equations (the normal equations), whose solution is:

import numpy as np
from scipy import stats

tea = np.array([98, 102, 101, 97, 102])
se = tea.std(ddof=1) / np.sqrt(len(tea))
low, high = stats.t.interval(0.95, df=4, loc=tea.mean(), scale=se)
print(round(low, 2), round(high, 2))

t, p = stats.ttest_1samp(tea, 103)
print(round(t, 3), round(p, 3))

hours = [1, 2, 3, 4, 5]
score = [52, 58, 65, 70, 80]
b1, b0 = np.polyfit(hours, score, 1)
print(round(b1, 2), round(b0, 2), round(np.corrcoef(hours, score)[0, 1], 3))

97.09 102.91
-2.86 0.046
6.8 44.6 0.994