..what's the difference between dqn and double dqn? why is it named like this? pros and…
- User
..what's the difference between dqn and double dqn? why is it named like this? pros and cons? be concise.
- Assistant
DQN vs. Double DQN (DDQN) – quick rundown
Aspect DQN Double DQN Core update 𝑌 = r + γ maxₐ′ Q_target(s′, a′) 𝑌 = r + γ Q_target(s′, argmaxₐ′ Q_online(s′, a′)) Key idea Same network selects and evaluates the max-value action ⇒ over-estimation bias. Use one network to select the action (argmax) and the other to evaluate it ⇒ reduces bias. Name origin “Deep Q-Network”. “Double” comes from classic “Double Q-learning” (Van Hasselt, 2010) that also splits selection & evaluation. Pros & cons (vs. vanilla DQN)
Pros
• Much lower over-estimation → more stable learning, better final scores on many Atari / RL benchmarks.
• Zero/near-zero extra compute — just a different target calculation.
• Same code base; drop-in change in most frameworks.Cons
• Slightly slower early learning because optimism is reduced.
• Gains not universal; on some simple tasks DQN ≈ DDQN.
• Still inherits other DQN limitations (e.g., sample inefficiency, sensitivity to hyper-params).That’s basically it: “double” just means “two value estimates” (selection vs. evaluation) to tame over-estimation.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.