Skip to content
TheChatArchiveThe Chat Archive
Claudeclaude-3-7-sonnet-20250219· AnthropicHugging Face LMArena Human Preference 140K

What are the best locally usable open ai models available given a 10gb vram gpu?

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with Geminigemini-2.5-pro’s answer.
  1. User

    What are the best locally usable open ai models available given a 10gb vram gpu?

  2. Claude Assistant

    For a 10GB VRAM GPU, you have several good locally usable open AI models to choose from:

    Mistral 7B or Mixtral 8x7B (quantized) - Great balance of performance and efficiency, with the quantized versions running well on your hardware.

    Llama 3 8B - Meta's latest small model that performs impressively for its size.

    Phi-3 Mini (4B) - Microsoft's compact but capable model that's very VRAM-efficient.

    RWKV-4 (7B) - A unique RNN-based architecture that can be more memory-efficient than transformer models.

    I'd personally recommend starting with Mistral 7B in a 4-bit quantized format - it offers excellent performance for the size and runs comfortably on 10GB VRAM with room for context window.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.