Optional exercise for people with lots of spare time.

(very hard) In this exercise, we shall construct an example of a sequence of random variables that satisfies the weak law of large numbers, but not the strong law. The distribution of Xi will have to depend on i, because otherwise both laws would be satisfied. As a preliminary, we need to prove a lemma, which is one of the Borel-Cantelli lemmas.

Suppose we have an infinite sequence of mutually independent events $A_1,
A_2, \dots$. Let $a_i = \Prob(A_i)$, and let r be a positive integer.

Find an expression for the probability that none of the Ai with i > r occur.
Use the fact that $x-1 \leq e^{-x}$ to show that

\begin{displaymath}\Prob(\text{No $A_i$\space with $i>r$\space occurs}) \leq e^{-\sum_{i=r}^{\infty} a_i}.

Prove that if $\sum_{i=1}^{\infty} a_i$ diverges, then

\begin{displaymath}\Prob(\text{infinitely many $A_i$\space occur}) = 1.

Now, let Xi be a sequence of mutually independent random variables such that for each positive integer $i\geq 2$,

\begin{displaymath}\Prob(X_i = i) = \frac{1}{2i\log i}, \quad
\Prob(X_i = -i) = \frac{1}{2i\log i}, \quad
\Prob(X_i = 0) = 1 - \frac{1}{i\log i}.

When i=1 we let X1 = 0 with probability 1. As usual we let $S_n =
X_1 + \dots + X_n$. Note that the mean of each Xi is 0.
Find the variance of Sn.
Show that the sequence $\langle X_i \rangle$ satisfies the weak law of large numbers. I.e. prove that for any $\epsilon > 0$

\begin{displaymath}\Prob(\left\vert \frac{S_n}{n} \right\vert \geq \epsilon) \rightarrow 0,

as n tends to infinity.
We now show that $\langle X_i \rangle$ does not satisfy the strong law of large numbers. Suppose that $S_n/n \rightarrow 0$. Then because

\begin{displaymath}\frac{X_n}{n} = \frac{S_n}{n} - \frac{n-1}{n}\frac{S_{n-1}}{n-1},

we know that $X_n/n \rightarrow 0$. From the definition of limits, we conclude that the inequality $\left\vert X_i \right\vert \geq \frac{1}{2} i$ can only be true for finitely many i.
Let Ai be the event $\left\vert X_i \right\vert \geq \frac{1}{2} i$. Find $\Prob(A_i)$. Show that $\sum_{i=1}^{\infty}\Prob(A_i)$ diverges (think integral test).
Prove that Ai occurs for infinitely many i.
Prove that

\begin{displaymath}\Prob(\frac{S_n}{n} \rightarrow 0) = 0,

and hence that the strong law of large numbers fails for the sequence $\langle X_i \rangle$.

David Maslen