Skip to content
TheChatArchiveThe Chat Archive
Grok 4.1 Fast· xaiHugging Face Frontier Reasoning Traces

Prove the law of large numbers.

  1. User

    Prove the law of large numbers.

  2. Assistant

    Elementary, my dear inquirer. The Law of Large Numbers (LLN) asserts that, under mild conditions, the sample average of i.i.d. random variables converges to their common expectation. I'll prove the Weak Law of Large Numbers (WLLN) using Chebyshev's inequality—a straightforward deduction requiring only finite mean and variance. (The Strong Law follows from more advanced machinery like Kolmogorov's criterion, but this suffices for most cases.)

    Statement (Weak LLN)

    Let ( {X_i}{i=1}^\infty ) be i.i.d. random variables with ( \mathbb{E}[X_i] = \mu ) and ( \mathrm{Var}(X_i) = \sigma^2 < \infty ). Define the sample average [ \bar{X}n = \frac{1}{n} \sum{i=1}^n X_i. ] Then, for every ( \epsilon > 0 ), [ \lim{n \to \infty} P(|\bar{X}_n - \mu| \geq \epsilon) = 0. ]

    Proof

    1. Expectation and variance of the average: [ \mathbb{E}[\bar{X}n] = \frac{1}{n} \sum{i=1}^n \mathbb{E}[X_i] = \mu, ] since the ( X_i ) are i.i.d. For the variance, [ \mathrm{Var}(\bar{X}n) = \mathrm{Var}\left( \frac{1}{n} \sum{i=1}^n X_i \right) = \frac{1}{n^2} \sum_{i=1}^n \mathrm{Var}(X_i) = \frac{\sigma^2}{n}, ] again by independence and identical variance.

    2. Apply Chebyshev's inequality: Chebyshev states that for any random variable ( Y ) with finite mean ( m ) and variance ( v < \infty ), and ( \epsilon > 0 ), [ P(|Y - m| \geq \epsilon) \leq \frac{v}{\epsilon^2}. ] Set ( Y = \bar{X}_n ), ( m = \mu ), ( v = \sigma^2 / n ): [ P(|\bar{X}_n - \mu| \geq \epsilon) \leq \frac{\mathrm{Var}(\bar{X}_n)}{\epsilon^2} = \frac{\sigma^2}{n \epsilon^2}. ]

    3. Take the limit: As ( n \to \infty ), ( \frac{\sigma^2}{n \epsilon^2} \to 0 ). Thus, [ \lim_{n \to \infty} P(|\bar{X}_n - \mu| \geq \epsilon) = 0. ]

    Q.E.D.

    This holds even without identical distributions (just i.i.d. with uniform bounded variance). For the Strong LLN (( \bar{X}_n \to \mu ) almost surely), finite expectation suffices under Kolmogorov's conditions—no variance needed. Care to see that proof, or apply it to a case?

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.