Skip to content
TheChatArchiveThe Chat Archive
Human-written demonstration (OpenAssistant volunteers)· OpenAssistant (human-written)Hugging Face OpenAssistant Conversations (oasst2)

What is the minimum data size needed to train a model that understands human languages??

In OpenAssistant data, “assistant” turns were written by human volunteers as demonstrations. They are not model outputs.
  1. User

    What is the minimum data size needed to train a model that understands human languages??

  2. Assistant

    The minimum data size needed to train a model that understands human languages can vary. It is highly dependent on the specific tasks.

    For instance, simple task like sentiment analysis model can be trained with a few thousand labeled data sets. On the other hand, for a more complex task like large language models, you would need millions or even billions of label data sets to achieve acceptable results.

  3. User

    How can I collect this data?

  4. Assistant

    If all goes well, you should be able to use the dataset that was used to train this model as it should be open source by then.

    If you would like to collect data yourself however, you would probably have to apply the help from others. Creating and labelling thousands of paragraphs on your own might be tiring pretty quickly and I would advise against it.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.