The law of large numbers is a precise theorem of probability theory. Well, two theorems actually.

The weak law of large numbers:

A sequence X1, X2, ... of independent identically distributed random variables with finite mean m and finite variance is given. Then the sequence Sn = (X1 + X2 + ...+ Xn)/n, for n = 1, 2, ... satisfies the following:
For any e > 0

P(|Sn - m| > e) → 0 as n → ∞.

The strong law of large numbers:

Under the same conditions as above Sn satisfies

P(Sn → m as n → ∞) = 1.


The two theorems look very similar, and sort of say that as you perform more and more experiments the mean of the sample will get closer to the expectation. They say it in different ways though, and getting your head round the difference can be tricky.
Roughly speaking SLLN says something about the behaviour of the particular sequences S1, S2, ... once we have decided the values for X1, X2, ..., while WLLN rather tells you something about the the probability distribution of Sn at different values of n.

The difference may be easier to understand if I give an example of a random sequence T1, T2, ... that satisfies the result of WLLN but not SLLN.
Let Y1, Y2, ... be sequence of independent random variables, such that Yk is uniformly distributed on the set {2k, 2k+1, ..., 2k+1-1}. Define Tn = 1 if n = Yk for some k and Tn = 0 otherwise.
For Tn thus defined we see that

P(Tn = 0) = 2-k

for 2k ≤ n < 2k+1, but that Tn always contains an infinite number of both ones and zeroes. Thus Tn satisfies the result of WLLN (with m = 0), but not the result of SLLN.
It follows that SLLN cannot be derived from WLLN. A shame really, since WLLN can be proved easily using Chebyshev's inequality, while SLLN is the result that we are really interested in.

It is, however, possible to derive WLLN from SLLN, which we might as well do while we're at it.
Suppose that a random sequence Sn satisfies the conditions of SLLN (with m = 0 for simplicity).
Given e > 0 let EN be the event that |Sn| < e for n ≥ N. We note that EN ⊆ EN+1 for all N and UNION(N = 0, ∞)(EN) ⊇ {Sn -> 0 as n -> ∞}, so limN→∞P(EN) ≥ P(Sn → 0 as n → ∞) = 1 (this is a basic result of measure theory). Also {|Sn| < e} ⊇ En for all n, so P(|Sn| < e) ≥ P(En).
Hence

limn→∞P(|Sn| < e) ≥ limn→∞P(En) ≥ 1

So Sn satisfies the result of WLLN with m = 0.

Thus we have shown that (unsurprsingly) the strong law of large numbers is a stronger result than the weak law of large numbers. (In the language of measure theory we have shown that if Xn → X almost surely then Xn → X in probability, but that the converse does not necessarily hold.)

Log in or registerto write something here or to contact authors.