Skip to content
TheChatArchiveThe Chat Archive
Qwenqwq-32b· Alibaba (Qwen)Hugging Face LMArena Human Preference 140K

list best llm for coding, which can be run with ollama on nvidia cuda with 4GB video ram

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with Qwenqwen3-235b-a22b-no-thinking’s answer.
  1. User

    list best llm for coding, which can be run with ollama on nvidia cuda with 4GB video ram

  2. Qwen Assistant

    Here’s a list of LLMs optimized for coding that can run on Ollama with a NVIDIA GPU (4GB VRAM), along with their key features and VRAM requirements:


    1. CodeLlama (7B)

    • VRAM Usage: ~3.5GB
    • Description:
      • Specifically trained for coding tasks (code generation, debugging, etc.).
      • Part of Meta’s Llama series, fine-tuned on large code datasets.
      • Supports multiple programming languages and code completion.
    • Why It’s Great:
      • One of the most advanced coding-focused LLMs.
      • Works well within 4GB VRAM with optimizations (e.g., FP16 precision).

    2. StarCoder (1B)

    • VRAM Usage: ~1.5GB
    • Description:
      • A smaller variant of the StarCoder family (trained on GitHub data).
      • Excels at code generation and natural language understanding.
    • Why It’s Great:
      • Lightweight and fast for smaller tasks.
      • Good for basic coding assistance with minimal VRAM.

    3. Pythia (2.8B)

    • VRAM Usage: ~2.5GB
    • Description:
      • A community-driven model from EleutherAI, trained on diverse code and text.
      • Versatile for coding and general NLP tasks.
    • Why It’s Great:
      • Affordable VRAM usage and strong performance for coding.
      • Available on Ollama and easy to configure.

    4. Llama2 (7B)

    • VRAM Usage: ~3.5GB
    • Description:
      • Meta’s Llama2 variant, fine-tuned for code and general tasks.
      • Supports code generation and natural language understanding.
    • Why It’s Great:
      • Well-rounded performance for coding and text tasks.
      • Similar VRAM requirements to CodeLlama but slightly less specialized.

    5. Mistral (7B)

    • VRAM Usage: ~3.5GB
    • Description:
      • A fast, lightweight model optimized for speed and efficiency.
      • Supports coding and natural language tasks.
    • Why It’s Great:
      • Low latency and good for quick coding assistance.

    6. Falcon (7B)

    • VRAM Usage: ~3.5GB
    • Description:
      • A versatile model from Technology Innovation Institute, trained on code and text.
      • Strong for code generation and multi-modal tasks.
    • Why It’s Great:
      • Flexible and performs well within 4GB VRAM.

    Tips for Running on Ollama with 4GB VRAM

    1. Use FP16 Precision: Enable mixed-precision (FP16) to reduce memory usage.
      • Example: ollama run <model> --model-args precision=fp16.
    2. Limit Context Length: Reduce max_new_tokens to avoid GPU memory spikes.
    3. Quantization: Use quantize (e.g., 4-bit quantization) to shrink the model further.
      • Example: ollama quantize <model> --bits 4.

    Why These Models?

    • CodeLlama is the top choice for coding-specific tasks due to its specialized training.
    • StarCoder (1B) is ideal for lightweight setups.
    • Pythia (2.8B) offers a balance between performance and memory.
    • All models listed are available on Ollama and can run on 4GB VRAM with optimizations.

    Let me know if you need help configuring Ollama! 😊

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.