Vicuna AI
- User
Vicuna AI
Assistant
Vicuna is an open-source chatbot developed by LMSYS, created by fine-tuning the Llama 2 model on user-shared conversations collected from ShareGPT[1][3]. It has gained attention for its impressive performance, with preliminary evaluations using GPT-4 as a judge showing that Vicuna-13B achieves more than 90% quality of OpenAI ChatGPT and Google Bard while outperforming other models like LLaMA and Stanford Alpaca in more than 90% of cases[3].
Key Features
- Open-source: The training and serving code, along with an online demo, are publicly available for non-commercial use[2][3].
- Cost-effective: The cost of training Vicuna-13B is estimated to be around $300[3].
- Multiple versions: Vicuna is available in different sizes (7B, 13B, 33B) and variants, with context sizes ranging from 2K to 16K tokens[8].
- Performance: Preliminary evaluations show Vicuna achieving competitive performance against proprietary models like ChatGPT and Bard[3].
Training and Architecture
Vicuna is fine-tuned from Llama 2 using approximately 125K conversations collected from ShareGPT.com[1][3]. The training process incorporates several improvements over previous models:
- Multi-turn conversations: The training loss is adjusted to account for multi-turn conversations.
- Memory optimizations: The max context length is expanded from 512 to 2048 tokens, utilizing gradient checkpointing and flash attention to manage increased GPU memory requirements.
- Cost reduction: Spot instances are used to reduce training costs[3].
Applications and Limitations
The primary intended use of Vicuna is for research on large language models and chatbots[1]. While it demonstrates impressive capabilities, it's important to note that, like other chatbots, Vicuna still faces limitations in areas such as basic math problems and coding abilities[3].
Availability and Usage
Vicuna can be accessed through various methods:
- Command-line interface
- APIs (OpenAI API, Huggingface API)
- Online demo at chat.lmsys.org[1][5]
For those interested in running Vicuna locally, it can be installed on a machine using either CPU or GPU, following the installation steps provided in the GitHub repository[5].
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.