QWEN3
- User
QWEN3
Assistant
Qwen3: Alibaba's Latest Advanced Language Model Family
Qwen3, released on April 28, 2025, is the latest generation of large language models from Alibaba's Qwen team[1][6]. This new family of models represents a significant advancement in natural language processing and multimodal capabilities, building upon the success of previous iterations like QwQ and Qwen2.5[3].
Model Variants and Architecture
Qwen3 comes in multiple sizes and configurations:
- Dense Models: Qwen3-0.6B, Qwen3-1.7B, Qwen3-4B, Qwen3-8B, Qwen3-14B, and Qwen3-32B[1][3]
- Mixture-of-Experts (MoE) Models:
- Qwen3-30B-A3B: 30 billion total parameters with 3 billion activated parameters[1][3][7]
- Qwen3-235B-A22B: 235 billion total parameters with 22 billion activated parameters[1][2][3]
The flagship model, Qwen3-235B-A22B, features 94 layers, 64 attention heads for queries and 4 for key-values, and 128 experts with 8 activated experts per token[2].
Key Innovations and Features
Hybrid Thinking Modes
One of Qwen3's most distinctive features is its dual reasoning approach[1][2][3]:
- Thinking Mode: Allows the model to reason step-by-step for complex problems requiring deeper thought
- Non-Thinking Mode: Provides quick, direct responses for simpler questions
Users can seamlessly switch between these modes to optimize performance for different tasks[5][7].
Enhanced Context Length
Qwen3 models natively support a context length of 32,768 tokens[2][5]. Using the YaRN method, this can be extended to 131,072 tokens, enabling processing of extremely long texts[2].
Multilingual Capabilities
The models support 119 languages and dialects, enabling international applications with strong capabilities for multilingual instruction following and translation[1][3][7].
Specialized Variants
Beyond the core models, the Qwen3 family includes specialized variants[4]:
- Qwen3-VL: Vision-Language model that integrates visual and textual data
- Qwen3-Math: Specialized for mathematical reasoning and problem-solving
- Qwen3-Audio: Handles audio processing tasks including transcription and spoken language understanding
Training Approach
Qwen3 was trained on approximately 36 trillion tokens, nearly double the amount used for Qwen2.5[1]. The training data included text from web sources, PDF documents, textbooks, question-answer pairs, and code snippets[1][6].
The pre-training followed a three-stage approach[1]:
- Initial pretraining on 30+ trillion tokens with 4K context length
- Additional training on 5 trillion tokens with increased STEM, coding, and reasoning content
- Context length extension to 32K tokens using high-quality long-context data
Performance
According to Alibaba, Qwen3 models demonstrate impressive performance on various benchmarks[1][6]:
- The flagship Qwen3-235B-A22B reportedly achieves competitive results compared to models like DeepSeek-R1, o1, o3-mini, Grok-3, and Gemini-2.5-Pro
- Qwen3-30B-A3B reportedly outperforms QwQ-32B despite using only 10% of the active parameters
- Even smaller models like Qwen3-4B can reportedly rival the performance of much larger previous-generation models like Qwen2.5-72B-Instruct
On specific benchmarks, Qwen3-235B-A22B purportedly outperforms OpenAI's o3-mini on Codeforces and AIME math benchmarks[6].
Deployment Options
Qwen3 models can be deployed through various frameworks[3][5][7]:
- SGLang:
python -m sglang.launch_server --model-path Qwen/Qwen3-8B --port 30000 --reasoning-parser qwen3 - vLLM:
vllm serve Qwen/Qwen3-8B --port 8000 --enable-reasoning --reasoning-parser deepseek_r1 - Ollama:
ollama run qwen3:8b(different sizes available) - Other options include LMStudio, MLX-LM, llama.cpp, and KTransformers
The models are also accessible through cloud providers like Alibaba Cloud Model Studio, Fireworks AI, and Hyperbolic[6].
Availability and Licensing
All Qwen3 dense models are available under the Apache 2.0 license[1]. They can be accessed via Hugging Face, ModelScope, and GitHub repositories[3].
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.