The

law of

large numbers is a precise

theorem of

probability theory. Well, two theorems actually.

**The weak law of large numbers:**

A sequence X_{1}, X_{2}, ... of independent identically distributed random variables with finite mean m and finite variance is given. Then the sequence S_{n} = (X_{1} + X_{2} + ...+ X_{n})/n, for n = 1, 2, ... satisfies the following:

For any e > 0

P(|S_{n} - m| > e) → 0 as n → ∞.

**The strong law of large numbers:**

Under the same conditions as above S_{n} satisfies

P(S_{n} → m as n → ∞) = 1.

The two theorems look very similar, and sort of say that as you perform more and more experiments the mean of the sample will get closer to the expectation. They say it in different ways though, and getting your head round the difference can be tricky.

Roughly speaking SLLN says something about the behaviour of the particular sequences S_{1}, S_{2}, ... once we have decided the values for X_{1}, X_{2}, ..., while WLLN rather tells you something about the the probability distribution of S_{n} at different values of n.

The difference may be easier to understand if I give an example of a random sequence T_{1}, T_{2}, ... that satisfies the result of WLLN but not SLLN.

Let Y_{1}, Y_{2}, ... be sequence of independent random variables, such that Y_{k} is uniformly distributed on the set {2^{k}, 2^{k}+1, ..., 2^{k+1}-1}.
Define T_{n} = 1 if n = Y_{k} for some k and T_{n} = 0 otherwise.

For T_{n} thus defined we see that

P(T_{n} = 0) = 2^{-k}

for 2^{k} ≤ n < 2^{k+1}, but that T_{n} always contains an infinite number of both ones and zeroes. Thus T_{n} satisfies the result of WLLN (with m = 0), but not the result of SLLN.

It follows that SLLN cannot be derived from WLLN. A shame really, since WLLN can be proved easily using Chebyshev's inequality, while SLLN is the result that we are really interested in.

It is, however, possible to derive WLLN from SLLN, which we might as well do while we're at it.

Suppose that a random sequence S_{n} satisfies the conditions of SLLN (with m = 0 for simplicity).

Given e > 0 let E_{N} be the event that |S_{n}| < e for n ≥ N. We note that E_{N} ⊆ E_{N+1} for all N and UNION(N = 0, ∞)(E_{N}) ⊇ {S_{n} -> 0 as n -> ∞}, so lim_{N→∞}P(E_{N}) ≥ P(S_{n} → 0 as n → ∞) = 1 (this is a basic result of measure theory). Also {|S_{n}| < e} ⊇ E_{n} for all n, so P(|S_{n}| < e) ≥ P(E_{n}).

Hence

lim_{n→∞}P(|S_{n}| < e) ≥ lim_{n→∞}P(E_{n}) ≥ 1

So S_{n} satisfies the result of WLLN with m = 0.

Thus we have shown that (unsurprsingly) the strong law of large numbers is a stronger result than the weak law of large numbers. (In the language of measure theory we have shown that if X_{n} → X almost surely then X_{n} → X in probability, but that the converse does not necessarily hold.)