Skip to content
The Chat Archive
Human-written demonstration (OpenAssistant volunteers)· OpenAssistant (human-written) OpenAssistant Conversations (oasst2)

What would be the best way to create a new generative model using novel techniques?

In OpenAssistant data, “assistant” turns were written by human volunteers as demonstrations. They are not model outputs.
  1. User

    What would be the best way to create a new generative model using novel techniques?

  2. Assistant

    Generative models models are still a relatively new development in machine learning research, and new techniques are developed frequently. A good starting point for coming up with new ideas will be to follow ongoing research by studying research papers and other publications. You may find inspiration for novel techniques by transferring knowledge from related fields. Another approach could be to try to apply methods that have been developed for one application to other use cases. Generative models are still in their infancy, so there is certainly a lot of room for improvement and possibilities for applications.

  3. User

    Can you give me a few examples of what can I start with if I'm interested in digging into text generation models?

  4. Assistant

    Certainly! Here are a few examples of what you could start with if you're interested in text generation models:

    1. GPT-2: GPT-2 is a generative pre-trained transformer model developed by OpenAI that can be used for a wide range of text generation tasks, from chatbots to creative writing. It has gained a lot of attention in the natural language processing community for its impressive text generation capabilities, but is a little outdated as it was released back in 2019.
    2. LLaMA: LLaMA is a recent foundation model that aims to be efficient and scalable, while still providing state-of-the-art performance on a range of language tasks. It was developed by a team of researchers from Facebook AI Research and the University of California, Berkeley.
    3. Alpaca-LoRA: Alpaca-LoRA is a recent adaptation of the Stanford Alpaca model that uses low-rank adaptation to improve the efficiency of fine-tuning. It was developed by a team of researchers from Stanford University and is based on the LLaMA model.
    4. Open Assistant: Open Assistant is a large language model that was developed by the ai community. It is intended to be a chat-based AI assistant that can answer a wide range of questions and carry out tasks based on natural language input. The good thing about Open Assistant is, that it doesn't belong to a central company and it's dataset as well as it's code is compleatly open source.

    These models have different strengths and weaknesses, so the best choice depends on your specific needs and interests. It's a good idea to read up on each model's architecture and capabilities before deciding which one to use.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.