Skip to content
TheChatArchiveThe Chat Archive
ChatGPT (model unspecified)· OpenAIHugging Face ShareLM (browser-extension contributions)

make the following a single tweet (from a thread of tweets). Note character limit We…

  1. User

    make the following a single tweet (from a thread of tweets). Note character limit

    We hypothesize that RL (and the KL regularization) indeed prevent from diverging from the base LLMe. But the responses we suggest are both very likely, given the prompt that explicitly asks for them. We just teach the model to prefer one of the two most probable answers. By itself, it understands to update those in other settings as well.

  2. Assistant

    RL (with KL regularization) keeps the model close to the base LLM. But our prompts ask for likely responses—just nudging it to prefer one over another. The model generalizes this preference elsewhere on its own.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.