Skip to content
TheChatArchiveThe Chat Archive
Perplexitysonar-pro-high· PerplexityHugging Face LMArena Search Arena 24K

show me the formula of KL divergence

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with Geminigemini-2.0-flash-grounding’s answer.
  1. User

    show me the formula of KL divergence

  2. Perplexity Assistant

    The formula for Kullback-Leibler (KL) divergence, which measures how one probability distribution ( Q ) diverges from a second, reference probability distribution ( P ), is given by:

    For Discrete Probability Distributions:

    [ D_{\text{KL}}(P \parallel Q) = \sum_{x \in X} P(x) \log \left( \frac{P(x)}{Q(x)} \right) ] Here:

    • ( P(x) ) and ( Q(x) ) are the probabilities of event ( x ) under distributions ( P ) and ( Q ), respectively.
    • ( X ) is the set of all possible events.

    For Continuous Probability Distributions:

    [ D_{\text{KL}}(P \parallel Q) = \int_{-\infty}^{\infty} p(x) \log \left( \frac{p(x)}{q(x)} \right) dx ] Here:

    • ( p(x) ) and ( q(x) ) are the probability density functions of ( P ) and ( Q ), respectively.

    Key Properties:

    • KL divergence is asymmetric, meaning ( D_{\text{KL}}(P \parallel Q) \neq D_{\text{KL}}(Q \parallel P) ).
    • It is always non-negative (( D_{\text{KL}} \geq 0 )) and equals zero only when ( P ) and ( Q ) are identical[1][2][3].

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.