Teach me MCMC with a simple example
- User
Teach me MCMC with a simple example
- Assistant
Markov Chain Monte Carlo (MCMC) Explained with a Simple Example
MCMC is a powerful technique used to sample from complex probability distributions, especially when direct sampling is difficult or impossible. Let's walk through a simple example to understand the basics.
Problem: Suppose we want to estimate the mean (
θ) of a normal distribution, but we can't sample directly from it. Instead, we have access to a likelihood function that tells us how likely our data is given a particularθ.Given:
- Data:
x = [1.2, 1.5, 0.8, 1.3](assumed to be generated from a normal distribution) - Prior distribution:
θ ~ Normal(0, 1)(our initial belief aboutθ) - Likelihood:
x | θ ~ Normal(θ, 0.2)(how likely the data is givenθ)
Goal: Sample from the posterior distribution
P(θ | x), which represents our updated belief aboutθafter seeing the data.MCMC Algorithm: Metropolis-Hastings
We'll use the Metropolis-Hastings algorithm, a popular MCMC method. Here's a step-by-step breakdown:
-
Initialize:
- Start with an initial guess for
θ, sayθ₀ = 1.0. - Choose a proposal distribution, e.g.,
Normal(θₜ, 0.1), whereθₜis the current value ofθ.
- Start with an initial guess for
-
Iterate: For each step
t: a. Propose a new value: Sampleθ' ~ Normal(θₜ, 0.1). b. Calculate acceptance ratio:α = min(1, (P(θ' | x) \* P(θₜ | θ')) / (P(θₜ | x) \* P(θ' | θₜ)))Where:
P(θ | x)is the posterior probability (proportional to likelihood * prior).P(θ' | θₜ)is the proposal probability (from the normal distribution).
c. Accept or reject:
- Generate a random number
u ~ Uniform(0, 1). - If
u < α, acceptθ'; otherwise, keepθₜ.
d. Update: Set
θₜ₊₁ = θ'(if accepted) orθₜ₊₁ = θₜ(if rejected). -
Repeat: Continue iterating until convergence (typically thousands of steps).
Python Implementation (Simplified):
import numpy as np # Data and parameters x = np.array([1.2, 1.5, 0.8, 1.3]) prior_mean, prior_std = 0, 1 likelihood_std = 0.2 # Initialize theta = 1.0 samples = [] # MCMC parameters n_iterations = 10000 proposal_std = 0.1 # Run MCMC for t in range(n_iterations): # Propose new value theta_proposal = np.random.normal(theta, proposal_std) # Calculate log-posterior (up to a constant) log_posterior_current = -0.5 * (theta - prior_mean)**2 / prior_std**2 - 0.5 * np.sum((x - theta)**2) / likelihood_std**2 log_posterior_proposal = -0.5 * (theta_proposal - prior_mean)**2 / prior_std**2 - 0.5 * np.sum((x - theta_proposal)**2) / likelihood_std**2 # Calculate acceptance ratio alpha = min(1, np.exp(log_posterior_proposal - log_posterior_current)) # Accept or reject if np.random.rand() < alpha: theta = theta_proposal # Store sample samples.append(theta) # Estimate posterior mean posterior_mean = np.mean(samples[1000:]) # Discard burn-in period print("Estimated posterior mean:", posterior_mean)Key Concepts:
- Markov Chain: Each sample depends only on the previous sample (Markov property).
- Monte Carlo: We use random sampling to approximate the posterior distribution.
- Convergence: MCMC chains need to run long enough to converge to the target distribution.
- Burn-in: Initial samples may not represent the posterior well, so we discard them.
This simple example demonstrates the basic idea behind MCMC. In practice, more sophisticated algorithms (like Hamiltonian Monte Carlo) and diagnostic tools are used to ensure convergence and efficiency.
- Data:
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.