i have a windows/ubunt dual boot laptop with 64GB RAM, AMD Ryzen 7 7745HX, RTX 4060 8GB.…
- User
i have a windows/ubunt dual boot laptop with 64GB RAM, AMD Ryzen 7 7745HX, RTX 4060 8GB. i want to use ollama and run an llm locally, which models would be best for general use and coding, to get the very best accuracy on my hardware
Assistant
Based on your powerful hardware (64GB RAM + RTX 4060 8GB), here are the best LLM recommendations for accuracy-focused general use and coding with Ollama, balancing quality and performance:
Top Recommendations
-
Mixtral 8x7B Instruct (47B parameters - MoE)
- Why:
- Best overall accuracy for coding/general tasks (beats many 70B models)
- Mixture-of-Experts design (only 12B active params per query)
- Excels at coding, reasoning, and multilingual tasks
- Hardware Fit:
- Runs well with 30-40 layers offloaded to GPU (Q4_K_M quantization)
- ~26GB RAM usage + full GPU utilization
- Ollama:
ollama run mixtral
- Why:
-
Llama 3 70B (70B parameters)
- Why:
- State-of-the-art general-purpose model (released April 2024)
- Best-in-class reasoning and instruction following
- Hardware Fit:
- Requires Q4_K_M quantization
- Offload 15-25 layers to GPU (fits in 64GB RAM + 8GB VRAM)
- Expect ~45-50GB RAM usage
- Ollama:
ollama run llama3:70b
- Why:
-
CodeLlama 34B Instruct (34B parameters)
- Why:
- Best coding-specific performance (Python/C++/docs)
- 16K context window for large codebases
- Hardware Fit:
- Offload 35-45 layers to GPU (Q4_K_M)
- ~20GB RAM usage
- Ollama:
ollama run codellama:34b
- Why:
Honorable Mentions
-
DeepSeek Coder 33B:
Top-tier for coding (better than CodeLlama 34B in some benchmarks)
Run:ollama run deepseek-coder:33b -
Qwen 1.5 72B:
Strong multilingual/generalist (requires Q4 quantization)
Run:ollama run qwen:72b
GPU Offloading Settings
For optimal GPU usage (RTX 4060 8GB):
# Example for Mixtral: OLLAMA_NUM_GPU=40 ollama run mixtral # Example for Llama 3 70B: OLLAMA_NUM_GPU=20 ollama run llama3:70bAdjust layers (
OLLAMA_NUM_GPU):- Start high (40 for Mixtral, 25 for 70B), reduce if OOM errors occur
- Monitor VRAM usage with
nvidia-smi
Quantization Advice
- Use Q4_K_M or Q5_K_M variants (best accuracy/size balance)
- Avoid Q2/Q3 - significant quality drop
Avoid These on Your Hardware
- 110B+ models (e.g., Goliath 120B) - too slow even with quantization
- Unquantized models - VRAM/RAM insufficient
- 7B-13B models - underutilizes your hardware (accuracy tradeoff)
Performance Expectations
Model Tokens/sec Quality Priority Mixtral 8x7B 10-15 🌟🌟🌟🌟🌟 (Best balance) Llama 3 70B 4-8 🌟🌟🌟🌟🌟 (Top accuracy) CodeLlama 34B 15-25 🌟🌟🌟🌟☆ (Coding specialist)
Start with Mixtral 8x7B for the best accuracy/speed balance. For heavy coding tasks, switch to CodeLlama 34B or DeepSeek Coder. Use Llama 3 70B when maximum reasoning quality is critical. Your hardware can comfortably handle these with proper layer offloading!
-
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.