Skip to content
The Chat Archive
Sign in
Human-written demonstration (OpenAssistant volunteers)· OpenAssistant (human-written) OpenAssistant Conversations (oasst2)

Is the free energy principle related to VQ-VAE?

In OpenAssistant data, “assistant” turns were written by human volunteers as demonstrations. They are not model outputs.
  1. User

    Is the free energy principle related to VQ-VAE?

  2. Assistant

    Yes, it is. Variational autoencoders learn to minimize reconstruction error by maximizing the evidence lower bound, known in variational Bayesian inference as the negative variational free energy. Gradient ascent of the evidence lower bound therefore minimizes surprisal, here referring to the negative log-probability of the generated distribution, and it's upper bound, the variational free energy.