Human-written demonstration (OpenAssistant volunteers)· OpenAssistant (human-written) OpenAssistant Conversations (oasst2)
palm e模型和gpt模型的区别
In OpenAssistant data, “assistant” turns were written by human volunteers as demonstrations. They are not model outputs.
- User
palm e模型和gpt模型的区别
- Assistant
palm e模型和gpt模型都是 Transformer 模型的变种,主要区别在于:
-
训练目标不同。GPT模型的训练目标是最大化下一个词的预测概率,是一种语言模型。而PLM模型的训练目标是最大化屏蔽过的词的预测概率,可以看作一种填充面具任务。PLM模型可以应用于更广的NLG任务,不仅限于下一词预测。
-
掩码策略不同。GPT模型没有掩码任何词,而PLM模型通过随机掩码的方式对输入文本进行处理,再进行预测。掩码可以包括替换为[MASK]标记,随机替换为其他词等。
-
预训练数据不同。GPT通常使用一般域的文本数据进行预训练,如Wikipedia、Book Corpus等。而PLM可以使用更广的文本类型,包括专业域文本、对话文本、代码等,因此可以应用于更多下游任务。
总体来说,GPT模型更注重语言模型能力的预训练,而PLM模型attempts更注重通用性预训练,能够更好地转移到其他NLG下游任务中。PLM比GPT模型有更强的适应性和泛化能力。
两者的区别可总结为:
• 训练目标:语言模型 VS 填充面具
• 掩码策略:无 VS 有
• 预训练数据:一般域 VS 更广域
• 适用范围:更偏向语言模型 VS 更广的NLG任务
目前主流的PLM模型有BERT、RoBERTa、XLNet、ELECTRA等,主流的GPT模型有GPT、GPT-2、GPT-3等。
-
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.
Identical first prompt
- Human-written demonstration (OpenAssistant volunteers)Compare side by side →
- Human-written demonstration (OpenAssistant volunteers)Compare side by side →
- Human-written demonstration (OpenAssistant volunteers)Compare side by side →
- Human-written demonstration (OpenAssistant volunteers)Compare side by side →
- Human-written demonstration (OpenAssistant volunteers)Compare side by side →
- Human-written demonstration (OpenAssistant volunteers)Compare side by side →
- Human-written demonstration (OpenAssistant volunteers)Compare side by side →
- Human-written demonstration (OpenAssistant volunteers)Compare side by side →