Skip to content
TheChatArchiveThe Chat Archive
Perplexitysonar-reasoning· PerplexityHugging Face LMArena Search Arena 24K

Vicuna AI

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with Perplexitysonar-pro-high’s answer.
  1. User

    Vicuna AI

  2. Perplexity Assistant

    Vicuna is an open-source chatbot developed by LMSYS, fine-tuned from Meta's Llama models using user-shared conversations collected from ShareGPT[1][3][5]. It is designed to provide high-quality responses comparable to proprietary models like ChatGPT and Google Bard, achieving over 90% of their quality in preliminary evaluations using GPT-4 as a judge[2][3][6].

    Key Features

    • Model Variants: Available in 7B, 13B, and 33B parameter sizes, with context lengths ranging from 2K to 16K tokens[8].
    • Training: Fine-tuned using supervised instruction tuning on ~125K ShareGPT conversations, optimized for multi-turn dialogue and long-context understanding[1][3][4].
    • Performance: Outperforms open-source models like Llama and Stanford Alpaca in most cases, with GPT-4 preferring Vicuna's responses in over 90% of comparisons[3][6].
    • Cost: Training costs are estimated at ~$300, making it accessible for research and non-commercial use[3][5].

    Availability

    • Code & Demo: Training and serving code are publicly available via GitHub, with an online demo accessible at chat.lmsys.org[3][5].
    • Deployment: Can be run locally on CPU/GPU, though not currently supported on Hugging Face's Inference API[1][5].

    Limitations

    • Struggles with basic math and coding tasks compared to proprietary models[3].
    • Non-commercial license restricts deployment in production environments[4][7].

    Vicuna serves as a benchmark for open-source chatbot research, enabling direct comparisons between models through its omnibus evaluation framework[7].

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.