Skip to content
TheChatArchiveThe Chat Archive
o3-2025-04-16· OpenAIHugging Face LMArena Human Preference 140K

..what's the difference between dqn and double dqn? why is it named like this? pros and…

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with Geminigemini-2.5-flash-preview-04-17’s answer.
  1. User

    ..what's the difference between dqn and double dqn? why is it named like this? pros and cons? be concise.

  2. Assistant

    DQN vs. Double DQN (DDQN) – quick rundown

    AspectDQNDouble DQN
    Core update𝑌 = r + γ maxₐ′ Q_target(s′, a′)𝑌 = r + γ Q_target(s′, argmaxₐ′ Q_online(s′, a′))
    Key ideaSame network selects and evaluates the max-value action ⇒ over-estimation bias.Use one network to select the action (argmax) and the other to evaluate it ⇒ reduces bias.
    Name origin“Deep Q-Network”.“Double” comes from classic “Double Q-learning” (Van Hasselt, 2010) that also splits selection & evaluation.

    Pros & cons (vs. vanilla DQN)

    Pros
    • Much lower over-estimation → more stable learning, better final scores on many Atari / RL benchmarks.
    • Zero/near-zero extra compute — just a different target calculation.
    • Same code base; drop-in change in most frameworks.

    Cons
    • Slightly slower early learning because optimism is reduced.
    • Gains not universal; on some simple tasks DQN ≈ DDQN.
    • Still inherits other DQN limitations (e.g., sample inefficiency, sensitivity to hyper-params).

    That’s basically it: “double” just means “two value estimates” (selection vs. evaluation) to tame over-estimation.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.