Skip to content
TheChatArchiveThe Chat Archive
Claudeclaude-opus-4-20250514-thinking-16k· AnthropicHugging Face LMArena Human Preference 140K

Is RLHF still used for fine tuning sota models?

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with Claudeclaude-3-5-sonnet-20241022’s answer.
  1. User

    Is RLHF still used for fine tuning sota models?

  2. Claude Assistant

    Yes, RLHF (Reinforcement Learning from Human Feedback) continues to be an important technique for fine-tuning state-of-the-art language models as of early 2025, though the landscape has evolved significantly.

    While RLHF remains widely used, researchers have developed several alternatives and improvements:

    Direct Preference Optimization (DPO) has gained traction as a simpler alternative that achieves similar results without the complexity of training a separate reward model and using RL algorithms. DPO directly optimizes the policy using preference data.

    Constitutional AI (CAI) and other self-supervision methods reduce reliance on human feedback by having models critique and revise their own outputs based on a set of principles.

    Iterative refinement approaches like RLAIF (Reinforcement Learning from AI Feedback) use AI systems to generate feedback, though these often still incorporate human oversight.

    Many leading models now use hybrid approaches - combining RLHF with other techniques rather than relying on it exclusively. The exact methods used by specific commercial models are often proprietary, but RLHF or RLHF-inspired techniques remain part of the standard toolkit for aligning language models with human preferences and values.

    The field continues to actively research more efficient and scalable alternatives to traditional RLHF, but it hasn't been completely replaced in practice.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.