Skip to content
TheChatArchiveThe Chat Archive
Perplexitysonar-reasoning-pro-high· PerplexityHugging Face LMArena Search Arena 24K

What is the most powerful LLM that will run at a decent speed on a Macbook Pro M4 24GB?

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with Perplexitysonar’s answer.
  1. User

    What is the most powerful LLM that will run at a decent speed on a Macbook Pro M4 24GB?

  2. Perplexity Assistant

    Running Large Language Models on M4 MacBook Pro with 24GB RAM

    When it comes to running LLMs locally on a MacBook Pro M4 with 24GB RAM, you need to balance model size, performance, and available memory. Based on extensive testing across various M4 configurations, there are several powerful options that will run at acceptable speeds.

    Performance Considerations

    For local LLM usage, most users consider 10 tokens per second as the minimum acceptable performance threshold. Below this speed, the experience becomes frustratingly slow[10]. With the M4 architecture's Neural Engine being 3x more powerful than the M1 generation, you can expect better performance than previous Apple Silicon chips[14].

    Top LLM Options for M4 MacBook Pro 24GB

    Phi-3 (14B model)

    The Phi-3 14B model requires approximately 7.9GB of memory[7], making it one of the most efficient models for its capability. It's a strong choice because:

    • It fits comfortably within your 24GB RAM constraint
    • Offers excellent reasoning capabilities
    • Provides a good balance of performance and power
    • Made by Microsoft and designed for efficiency on consumer hardware[5]

    Qwen 2.5 (14B model)

    Qwen 2.5 14B requires around 9.0GB of memory[7] and offers excellent performance for coding and general tasks:

    • Pretrained on over 20 trillion tokens[2]
    • Particularly strong at code generation and debugging
    • Efficient architecture designed for low-latency responses
    • Available through Hugging Face and ModelScope[2][5]

    LLaMA 3.1 (8B model)

    The LLaMA 3.1 8B model with fp16 quantization requires approximately 16GB[7], leaving enough headroom for your system to operate:

    • Strong performance for general-purpose chatbots and text generation[13]
    • 128,000 token context window[5]
    • Part of the widely supported LLaMA "herd"[1]
    • Compatible with tools like Ollama and llama.cpp for optimized performance[13]

    Falcon 3 (10B model)

    Falcon 3 10B provides impressive speed and capability for its size:

    • Runs at approximately 55 tokens per second on M4 chips[10]
    • Good for data summarization and research queries[13]
    • Supports Hugging Face Transformers with Apple Silicon optimizations[13]

    Performance Expectations

    Based on benchmark data from M4 devices, you can expect:

    • Smaller models (1-10B parameters): 55-200 tokens per second[10]
    • Medium models (10-20B parameters): 20-55 tokens per second[10]
    • Larger models (30B+ parameters): Below 10 tokens per second in most cases[10]

    Tests on the M4 Pro MacBook show that models like Qwen 2.5 32B (which uses 20GB RAM) run at around 20 tokens per second[10], but this would be pushing the limits of your 24GB configuration.

    Optimization Tips

    To get the best performance from your M4 MacBook Pro:

    1. Use quantized models (like q4_K_M or q8_0) to reduce memory usage[13]
    2. Enable Metal acceleration for GPU computation[13]
    3. Use llama.cpp for LLaMA models, which offers excellent optimization for Apple Silicon[13][15]
    4. Close other memory-intensive applications when running larger models
    5. Consider external storage for model files to preserve system memory[13]

    Conclusion

    For a 24GB M4 MacBook Pro, the most powerful LLM that will run at a decent speed would be either the Phi-3 14B or Qwen 2.5 14B models. Both offer an excellent balance of capability and performance while fitting comfortably within your memory constraints. The LLaMA 3.1 8B model is also an excellent choice if you need the longer context window, though it will use more of your available RAM.

    If you're willing to sacrifice some speed, you might be able to run Gemma 2 27B (requiring around 16GB)[7], but it would likely operate near the lower threshold of acceptable performance.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.

Identical first prompt