Skip to content
The Chat Archive
command-a-03-2025· Cohere LMArena Human Preference 140K

Teach me MCMC with a simple example

  1. User

    Teach me MCMC with a simple example

  2. Assistant

    Markov Chain Monte Carlo (MCMC) Explained with a Simple Example

    MCMC is a powerful technique used to sample from complex probability distributions, especially when direct sampling is difficult or impossible. Let's walk through a simple example to understand the basics.

    Problem: Suppose we want to estimate the mean (θ) of a normal distribution, but we can't sample directly from it. Instead, we have access to a likelihood function that tells us how likely our data is given a particular θ.

    Given:

    • Data: x = [1.2, 1.5, 0.8, 1.3] (assumed to be generated from a normal distribution)
    • Prior distribution: θ ~ Normal(0, 1) (our initial belief about θ)
    • Likelihood: x | θ ~ Normal(θ, 0.2) (how likely the data is given θ)

    Goal: Sample from the posterior distribution P(θ | x), which represents our updated belief about θ after seeing the data.

    MCMC Algorithm: Metropolis-Hastings

    We'll use the Metropolis-Hastings algorithm, a popular MCMC method. Here's a step-by-step breakdown:

    1. Initialize:

      • Start with an initial guess for θ, say θ₀ = 1.0.
      • Choose a proposal distribution, e.g., Normal(θₜ, 0.1), where θₜ is the current value of θ.
    2. Iterate: For each step t: a. Propose a new value: Sample θ' ~ Normal(θₜ, 0.1). b. Calculate acceptance ratio:

      α = min(1, (P(θ' | x) \* P(θₜ | θ')) / (P(θₜ | x) \* P(θ' | θₜ)))
      

      Where:

      • P(θ | x) is the posterior probability (proportional to likelihood * prior).
      • P(θ' | θₜ) is the proposal probability (from the normal distribution).

      c. Accept or reject:

      • Generate a random number u ~ Uniform(0, 1).
      • If u < α, accept θ'; otherwise, keep θₜ.

      d. Update: Set θₜ₊₁ = θ' (if accepted) or θₜ₊₁ = θₜ (if rejected).

    3. Repeat: Continue iterating until convergence (typically thousands of steps).

    Python Implementation (Simplified):

    import numpy as np
    
    # Data and parameters
    x = np.array([1.2, 1.5, 0.8, 1.3])
    prior_mean, prior_std = 0, 1
    likelihood_std = 0.2
    
    # Initialize
    theta = 1.0
    samples = []
    
    # MCMC parameters
    n_iterations = 10000
    proposal_std = 0.1
    
    # Run MCMC
    for t in range(n_iterations):
        # Propose new value
        theta_proposal = np.random.normal(theta, proposal_std)
        
        # Calculate log-posterior (up to a constant)
        log_posterior_current = -0.5 * (theta - prior_mean)**2 / prior_std**2 - 0.5 * np.sum((x - theta)**2) / likelihood_std**2
        log_posterior_proposal = -0.5 * (theta_proposal - prior_mean)**2 / prior_std**2 - 0.5 * np.sum((x - theta_proposal)**2) / likelihood_std**2
        
        # Calculate acceptance ratio
        alpha = min(1, np.exp(log_posterior_proposal - log_posterior_current))
        
        # Accept or reject
        if np.random.rand() < alpha:
            theta = theta_proposal
        
        # Store sample
        samples.append(theta)
    
    # Estimate posterior mean
    posterior_mean = np.mean(samples[1000:])  # Discard burn-in period
    print("Estimated posterior mean:", posterior_mean)
    

    Key Concepts:

    • Markov Chain: Each sample depends only on the previous sample (Markov property).
    • Monte Carlo: We use random sampling to approximate the posterior distribution.
    • Convergence: MCMC chains need to run long enough to converge to the target distribution.
    • Burn-in: Initial samples may not represent the posterior well, so we discard them.

    This simple example demonstrates the basic idea behind MCMC. In practice, more sophisticated algorithms (like Hamiltonian Monte Carlo) and diagnostic tools are used to ensure convergence and efficiency.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.