What are the different sizes in the Yi 1.5 LLM group?
- User
What are the different sizes in the Yi 1.5 LLM group?
Assistant
The Yi 1.5 language model group consists of three primary model sizes, each available in various configurations to suit different use cases and hardware requirements:
Model Sizes
The Yi 1.5 model family comes in three parameter sizes:
- 34B (34 billion parameters): The largest and most capable model in the Yi 1.5 lineup[2]
- 9B (9 billion parameters): A medium-sized model that balances performance and resource requirements[2]
- 6B (6 billion parameters): The smallest variant, designed to be more accessible for deployment on consumer hardware[2]
Context Window Variants
Each of these model sizes comes with different context window capabilities:
- Standard (4K tokens): The base context length for all Yi 1.5 models[4][8]
- 16K tokens: Extended context window versions available for both the 34B and 9B models[1][7]
- 32K tokens: Larger context window variants for the 34B and 9B models[4][7]
Model Variants
For each size and context window combination, Yi 1.5 offers:
- Base models: Pre-trained foundation models that complete text but aren't specifically optimized for chat or instruction following[4][8]
- Chat models: Fine-tuned versions optimized for conversational applications[4][8]
Performance Characteristics
- The Yi-1.5-34B model is described as performing on par with or exceeding larger models in some benchmarks[4][8]
- The Yi-1.5-9B model is positioned as the top performer among similarly sized open-source models[4][8]
- All Yi 1.5 models were trained on approximately 3.6 trillion tokens and then continuously pre-trained with a high-quality corpus of 500 billion tokens[4][8]
- The models were further fine-tuned on 3 million diverse training samples[4][6]
Technical Implementation
The models are available in various formats for different deployment scenarios:
- PyTorch format with options for different quantization levels (4-bit, 8-bit, none)[3]
- GGUF format with multiple quantization options for efficient deployment[3][6]
- Support for different inference engines including vLLM, Transformers, and llama.cpp[3]
Unlike the original Yi model which supported context lengths up to 200,000 tokens, the Yi 1.5 series maxes out at 32K tokens for its longest context variant[7].
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.