Vicuna AI
- User
Vicuna AI
Assistant
Vicuna is an open-source chatbot developed by LMSYS, fine-tuned from Meta's Llama models using user-shared conversations collected from ShareGPT[1][3][5]. It is designed to provide high-quality responses comparable to proprietary models like ChatGPT and Google Bard, achieving over 90% of their quality in preliminary evaluations using GPT-4 as a judge[2][3][6].
Key Features
- Model Variants: Available in 7B, 13B, and 33B parameter sizes, with context lengths ranging from 2K to 16K tokens[8].
- Training: Fine-tuned using supervised instruction tuning on ~125K ShareGPT conversations, optimized for multi-turn dialogue and long-context understanding[1][3][4].
- Performance: Outperforms open-source models like Llama and Stanford Alpaca in most cases, with GPT-4 preferring Vicuna's responses in over 90% of comparisons[3][6].
- Cost: Training costs are estimated at ~$300, making it accessible for research and non-commercial use[3][5].
Availability
- Code & Demo: Training and serving code are publicly available via GitHub, with an online demo accessible at chat.lmsys.org[3][5].
- Deployment: Can be run locally on CPU/GPU, though not currently supported on Hugging Face's Inference API[1][5].
Limitations
- Struggles with basic math and coding tasks compared to proprietary models[3].
- Non-commercial license restricts deployment in production environments[4][7].
Vicuna serves as a benchmark for open-source chatbot research, enabling direct comparisons between models through its omnibus evaluation framework[7].
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.