Skip to content
TheChatArchiveThe Chat Archive
DeepSeekdeepseek-r1-0528· DeepSeekHugging Face LMArena Human Preference 140K

i have a windows/ubunt dual boot laptop with 64GB RAM, AMD Ryzen 7 7745HX, RTX 4060 8GB.…

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with Qwenqwen3-30b-a3b’s answer.
  1. User

    i have a windows/ubunt dual boot laptop with 64GB RAM, AMD Ryzen 7 7745HX, RTX 4060 8GB. i want to use ollama and run an llm locally, which models would be best for general use and coding, to get the very best accuracy on my hardware

  2. DeepSeek Assistant

    Based on your powerful hardware (64GB RAM + RTX 4060 8GB), here are the best LLM recommendations for accuracy-focused general use and coding with Ollama, balancing quality and performance:


    Top Recommendations

    1. Mixtral 8x7B Instruct (47B parameters - MoE)

      • Why:
        • Best overall accuracy for coding/general tasks (beats many 70B models)
        • Mixture-of-Experts design (only 12B active params per query)
        • Excels at coding, reasoning, and multilingual tasks
      • Hardware Fit:
        • Runs well with 30-40 layers offloaded to GPU (Q4_K_M quantization)
        • ~26GB RAM usage + full GPU utilization
      • Ollama: ollama run mixtral
    2. Llama 3 70B (70B parameters)

      • Why:
        • State-of-the-art general-purpose model (released April 2024)
        • Best-in-class reasoning and instruction following
      • Hardware Fit:
        • Requires Q4_K_M quantization
        • Offload 15-25 layers to GPU (fits in 64GB RAM + 8GB VRAM)
        • Expect ~45-50GB RAM usage
      • Ollama: ollama run llama3:70b
    3. CodeLlama 34B Instruct (34B parameters)

      • Why:
        • Best coding-specific performance (Python/C++/docs)
        • 16K context window for large codebases
      • Hardware Fit:
        • Offload 35-45 layers to GPU (Q4_K_M)
        • ~20GB RAM usage
      • Ollama: ollama run codellama:34b

    Honorable Mentions

    • DeepSeek Coder 33B:
      Top-tier for coding (better than CodeLlama 34B in some benchmarks)
      Run: ollama run deepseek-coder:33b

    • Qwen 1.5 72B:
      Strong multilingual/generalist (requires Q4 quantization)
      Run: ollama run qwen:72b


    GPU Offloading Settings

    For optimal GPU usage (RTX 4060 8GB):

    # Example for Mixtral:
    OLLAMA_NUM_GPU=40 ollama run mixtral
    
    # Example for Llama 3 70B:
    OLLAMA_NUM_GPU=20 ollama run llama3:70b
    

    Adjust layers (OLLAMA_NUM_GPU):

    • Start high (40 for Mixtral, 25 for 70B), reduce if OOM errors occur
    • Monitor VRAM usage with nvidia-smi

    Quantization Advice

    • Use Q4_K_M or Q5_K_M variants (best accuracy/size balance)
    • Avoid Q2/Q3 - significant quality drop

    Avoid These on Your Hardware

    • 110B+ models (e.g., Goliath 120B) - too slow even with quantization
    • Unquantized models - VRAM/RAM insufficient
    • 7B-13B models - underutilizes your hardware (accuracy tradeoff)

    Performance Expectations

    ModelTokens/secQuality Priority
    Mixtral 8x7B10-15🌟🌟🌟🌟🌟 (Best balance)
    Llama 3 70B4-8🌟🌟🌟🌟🌟 (Top accuracy)
    CodeLlama 34B15-25🌟🌟🌟🌟☆ (Coding specialist)

    Start with Mixtral 8x7B for the best accuracy/speed balance. For heavy coding tasks, switch to CodeLlama 34B or DeepSeek Coder. Use Llama 3 70B when maximum reasoning quality is critical. Your hardware can comfortably handle these with proper layer offloading!

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.