what is the best open source model to fine tune locally on chats about coding
- User
what is the best open source model to fine tune locally on chats about coding
Assistant
For fine-tuning locally on coding conversations, here are the top open source options:
Llama 3.1 (8B or 70B) - Meta's latest models have shown strong coding performance and are popular for fine-tuning. The 8B version is manageable on consumer GPUs (24GB VRAM), while 70B requires more substantial hardware but offers better base performance.
DeepSeek Coder V2 - Specifically designed for code, available in sizes from 1.3B to 33B parameters. The 6.7B and 16B versions offer a good balance of performance and hardware requirements. It's trained on a large corpus of code and has shown excellent results when fine-tuned on domain-specific coding tasks.
CodeLlama - Meta's code-specific variant of Llama 2, available in 7B, 13B, and 34B sizes. While older than Llama 3.1, it's still effective and has extensive community support and fine-tuning guides.
Mistral 7B - A strong general-purpose model that handles code well and is very efficient for its size. Easy to fine-tune on consumer hardware and has good instruction-following capabilities.
For your specific use case of coding chats, I'd recommend starting with either Llama 3.1 8B or DeepSeek Coder 6.7B. Both can be fine-tuned using tools like:
- Axolotl or Unsloth for efficient fine-tuning
- LoRA/QLoRA to reduce memory requirements
- Datasets formatted in a conversational structure (like ShareGPT format)
The choice depends on your hardware constraints and whether you want a model specialized for code (DeepSeek) or a more general model that's also strong at coding (Llama 3.1).
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.