Skip to content
TheChatArchiveThe Chat Archive
ClaudeClaude Haiku 4.5· anthropicHugging Face Frontier Reasoning Traces

Outline the central limit theorem proof using characteristic functions. Show the…

  1. User

    Outline the central limit theorem proof using characteristic functions. Show the characteristic function of the normalized sum approaches the Gaussian form by Taylor expanding the log.

  2. Claude Assistant

    This is a great question about proving the Central Limit Theorem (CLT) using characteristic functions. I'll provide a comprehensive outline of this elegant proof.

    Let me structure this clearly:

    1. Setup and definitions
    2. Key properties of characteristic functions
    3. The main proof using Taylor expansion of the logarithm
    4. Convergence to the Gaussian form

    Let me work through this carefully.

    Setup

    Let X1,X2,X3,…X_1, X_2, X_3, \ldots be i.i.d. random variables with:

    • Mean μ=E[Xi]\mu = E[X_i]
    • Variance σ2=Var(Xi)\sigma^2 = \text{Var}(X_i), where 0<σ2<∞0 < \sigma^2 < \infty

    Define the normalized sum: Sn=1n∑i=1nXi−μσS_n = \frac{1}{\sqrt{n}} \sum_{i=1}^{n} \frac{X_i - \mu}{\sigma}

    We want to show that Sn→dN(0,1)S_n \xrightarrow{d} N(0,1).

    Key Properties of Characteristic Functions

    The characteristic function of a random variable YY is: ϕY(t)=E[eitY]\phi_Y(t) = E[e^{itY}]

    Important property: If Yn→dYY_n \xrightarrow{d} Y, then ϕYn(t)→ϕY(t)\phi_{Y_n}(t) \to \phi_Y(t) for all t∈Rt \in \mathbb{R}.

    The converse is also true (continuity theorem).

    Proof

    Let Zi=Xi−μσZ_i = \frac{X_i - \mu}{\sigma}, so:

    • E[Zi]=0E[Z_i] = 0
    • Var(Zi)=1\text{Var}(Z_i) = 1

    Then: Sn=1n∑i=1nZiS_n = \frac{1}{\sqrt{n}} \sum_{i=1}^{n} Z_i

    ϕSn(t)=E[exp⁡(it1n∑i=1nZi)]\phi_{S_n}(t) = E\left[\exp\left(it \frac{1}{\sqrt{n}} \sum_{i=1}^{n} Z_i\right)\right]

    By independence of the ZiZ_i: ϕSn(t)=∏i=1nϕZi(tn)\phi_{S_n}(t) = \prod_{i=1}^{n} \phi_{Z_i}\left(\frac{t}{\sqrt{n}}\right)

    Since the ZiZ_i are identically distributed with characteristic function ϕZ\phi_Z: ϕSn(t)=[ϕZ(tn)]n\phi_{S_n}(t) = \left[\phi_Z\left(\frac{t}{\sqrt{n}}\right)\right]^n

    To analyze this, I'll take the logarithm and expand ϕZ(u)\phi_Z(u) around u=0u = 0. Using the Taylor series with E[Z]=0E[Z] = 0 and Var(Z)=1\text{Var}(Z) = 1: ϕZ(u)=E[eiuZ]=1+iuE[Z]−u22E[Z2]+O(u3)=1−u22+O(u3)\phi_Z(u) = E[e^{iuZ}] = 1 + iuE[Z] - \frac{u^2}{2}E[Z^2] + O(u^3) = 1 - \frac{u^2}{2} + O(u^3)

    Substituting u=tnu = \frac{t}{\sqrt{n}}: ϕZ(tn)=1−t22n+O(1n3/2)\phi_Z\left(\frac{t}{\sqrt{n}}\right) = 1 - \frac{t^2}{2n} + O\left(\frac{1}{n^{3/2}}\right)

    For the logarithm of the characteristic function: log⁡ϕSn(t)=nlog⁡ϕZ(tn)=nlog⁡(1−t22n+O(1n3/2))\log \phi_{S_n}(t) = n \log \phi_Z\left(\frac{t}{\sqrt{n}}\right) = n \log\left(1 - \frac{t^2}{2n} + O\left(\frac{1}{n^{3/2}}\right)\right)

    Using log⁡(1+x)=x−x22+O(x3)\log(1 + x) = x - \frac{x^2}{2} + O(x^3) with x=−t22nx = -\frac{t^2}{2n}, the logarithm simplifies to −t22-\frac{t^2}{2} as n→∞n \to \infty, which is the characteristic function of the standard normal distribution. t)→−t22t) \to -\frac{t^2}{2}

    so ϕSn(t)→e−t2/2\phi_{S_n}(t) \to e^{-t^2/2}, which is the characteristic function of a standard normal distribution. By the continuity theorem, this means SnS_n converges in distribution to N(0,1)N(0,1), establishing the Central Limit Theorem.

    The proof hinges on expressing the normalized sum's characteristic function as a power, expanding it via Taylor series around zero, leveraging the zero mean and unit variance properties, then applying the logarithmic expansion to isolate the −t2/2-t^2/2 term that dominates the limit.

    Central Limit Theorem Proof via Characteristic Functions

    Setup

    Let X1,X2,…X_1, X_2, \ldots be i.i.d. random variables with E[Xi]=μE[X_i] = \mu and Var(Xi)=σ2<∞\text{Var}(X_i) = \sigma^2 < \infty.

    Define the normalized sum: Sn=1n∑i=1nXi−μσS_n = \frac{1}{\sqrt{n}}\sum_{i=1}^{n} \frac{X_i - \mu}{\sigma}

    Goal: Show Sn→dN(0,1)S_n \xrightarrow{d} N(0,1) by proving ϕSn(t)→e−t2/2\phi_{S_n}(t) \to e^{-t^2/2}


    Step 1: Characteristic Function of the Normalized Sum

    Let Zi=(Xi−μ)/σZ_i = (X_i - \mu)/\sigma with E[Zi]=0E[Z_i] = 0 and Var(Zi)=1\text{Var}(Z_i) = 1.

    ϕSn(t)=E[eitSn]=E[exp⁡(itn∑i=1nZi)]\phi_{S_n}(t) = E\left[e^{itS_n}\right] = E\left[\exp\left(\frac{it}{\sqrt{n}}\sum_{i=1}^{n} Z_i\right)\right]

    By independence: ϕSn(t)=∏i=1nϕZ(tn)=[ϕZ(tn)]n\phi_{S_n}(t) = \prod_{i=1}^{n} \phi_Z\left(\frac{t}{\sqrt{n}}\right) = \left[\phi_Z\left(\frac{t}{\sqrt{n}}\right)\right]^n

    where ϕZ\phi_Z is the characteristic function of Z1Z_1.


    Step 2: Taylor Expand Around Zero

    For small uu: ϕZ(u)=E[eiuZ]=E[∑k=0∞(iuZ)kk!]\phi_Z(u) = E[e^{iuZ}] = E\left[\sum_{k=0}^{\infty} \frac{(iuZ)^k}{k!}\right]

    Using E[Z]=0E[Z] = 0, E[Z2]=1E[Z^2] = 1:

    ϕZ(u)=1+iuE[Z]⏟0+(iu)22E[Z2]+(iu)36E[Z3]+O(u4)\phi_Z(u) = 1 + \underbrace{iuE[Z]}_{0} + \frac{(iu)^2}{2}E[Z^2] + \frac{(iu)^3}{6}E[Z^3] + O(u^4)

    =1−u22+O(u3)= 1 - \frac{u^2}{2} + O(u^3)

    Substituting u=t/nu = t/\sqrt{n}: ϕZ(tn)=1−t22n+O(n−3/2)\phi_Z\left(\frac{t}{\sqrt{n}}\right) = 1 - \frac{t^2}{2n} + O(n^{-3/2})


    Step 3: Take the Logarithm

    log⁡ϕSn(t)=nlog⁡ϕZ(tn)\log \phi_{S_n}(t) = n \log \phi_Z\left(\frac{t}{\sqrt{n}}\right)

    =nlog⁡(1−t22n+O(n−3/2))= n \log\left(1 - \frac{t^2}{2n} + O(n^{-3/2})\right)

    Using log⁡(1+x)=x−x22+⋯\log(1 + x) = x - \frac{x^2}{2} + \cdots with x=−t22n+O(n−3/2)x = -\frac{t^2}{2n} + O(n^{-3/2}):

    log⁡(1+x)=(−t22n)−12(−t22n)2+O(n−3/2)\log(1 + x) = \left(-\frac{t^2}{2n}\right) - \frac{1}{2}\left(-\frac{t^2}{2n}\right)^2 + O(n^{-3/2})

    =−t22n+O(n−2)= -\frac{t^2}{2n} + O(n^{-2})


    Step 4: Convergence to Gaussian Form

    log⁡ϕSn(t)=n[−t22n+O(n−2)]=−t22+O(n−1)\log \phi_{S_n}(t) = n\left[-\frac{t^2}{2n} + O(n^{-2})\right] = -\frac{t^2}{2} + O(n^{-1})

    As n→∞n \to \infty: log⁡ϕSn(t)→−t22\log \phi_{S_n}(t) \to -\frac{t^2}{2}

    ϕSn(t)→e−t2/2\boxed{\phi_{S_n}(t) \to e^{-t^2/2}}


    Step 5: Apply Continuity Theorem

    The characteristic function of N(0,1)N(0,1) is ϕ(t)=e−t2/2\phi(t) = e^{-t^2/2}.

    By the continuity theorem: If ϕn(t)→ϕ(t)\phi_n(t) \to \phi(t) for all tt, then Sn→dXS_n \xrightarrow{d} X where XX has characteristic function ϕ\phi.

    Therefore: Sn→dN(0,1)\boxed{S_n \xrightarrow{d} N(0,1)}


    Key Insights

    AspectInsight
    IndependenceAllows product form for characteristic function
    Zero meanEliminates linear term in Taylor expansion
    Unit varianceMakes u2u^2 coefficient exactly −1/2-1/2
    Log linearizationConverts product →\to sum, scaling becomes additive
    nn cancellationThe nn factor in the log cancels the 1/n1/n from t2/2nt^2/2n

    The elegance: The O(n−1)O(n^{-1}) error vanishes because all higher order terms in the Taylor expansion have additional n−1n^{-1} factors that fail to survive multiplication by nn in the logarithm.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.