Skip to content
TheChatArchiveThe Chat Archive
GPT 5.1 Codex Max· openaiHugging Face Frontier Reasoning Traces

Explain the Metropolis-Hastings acceptance ratio. Why does the proposal density cancel…

  1. User

    Explain the Metropolis-Hastings acceptance ratio. Why does the proposal density cancel if symmetric? Explain how this allows sampling from posteriors without knowing the normalizing constant.

  2. Assistant

    Explaining Metropolis-Hastings acceptance ratio

    The Metropolis-Hastings acceptance ratio is defined as α(x,y) = min(1, [π(y) q(y,x)] / [π(x) q(x,y)]), ensuring detailed balance and that the Markov chain has π as its stationary distribution. When the proposal distribution q is symmetric—meaning q(x,y) = q(y,x)—these terms cancel, simplifying α to min(1, π(y)/π(x)). Since the target π is often known only up to a normalization constant, this ratio eliminates that unknown constant, allowing calculation of acceptance probabilities using unnormalized densities. This is why Metropolis-Hastings can sample from complex posteriors without knowing their normalizing constant. In Metropolis–Hastings you build a Markov chain with transition kernel [ K(x,\mathrm d y)=q(x,\mathrm d y),\alpha(x,y)+\Big(1-\int q(x,\mathrm d z),\alpha(x,z)\Big)\delta_x(\mathrm d y), ] where (q(x,\cdot)) is your proposal density and (\alpha(x,y)) is the probability of accepting a proposed move from (x) to (y).

    The form of (\alpha) is chosen so that the chain is reversible with respect to the target density (\pi), i.e. it satisfies detailed balance [ \pi(x),q(x,y),\alpha(x,y)=\pi(y),q(y,x),\alpha(y,x). ] The largest choice in ([0,1]) that satisfies this symmetry is [ \alpha(x,y)=\min!\left{1,\frac{\pi(y),q(y,x)}{\pi(x),q(x,y)}\right}. ]

    If the proposal is symmetric, (q(x,y)=q(y,x)), the (q)-terms drop out and you recover the original Metropolis acceptance rule [ \alpha(x,y)=\min!\left{1,\frac{\pi(y)}{\pi(x)}\right}. ]

    Because the acceptance probability only ever involves a ratio of (\pi) evaluated at two points, any constant factor common to (\pi(y)) and (\pi(x)) cancels as well. In a Bayesian context the posterior has the form [ \pi(\theta\mid \text{data})=\frac{L(\text{data}\mid\theta),p(\theta)}{p(\text{data})}, ] where the evidence (p(\text{data})) is usually intractable. In the Metropolis–Hastings ratio (\pi(y)/\pi(x)=\big[L(\text{data}\mid y)p(y)\big]/\big[L(\text{data}\mid x)p(x)\big]), the unknown normalising constant (p(\text{data})) cancels, so you can compute (\alpha) using the unnormalised posterior and still obtain a chain with the correct stationary distribution. This is what makes M–H practical for sampling from complicated posteriors.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.