Skip to content
TheChatArchiveThe Chat Archive
Perplexitysonar-reasoning-pro-high· PerplexityHugging Face LMArena Search Arena 24K

Recommend me the best edge TPU or other portable solution to run a Whisper +…

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with gpt-4o-search-preview-high-loc’s answer.
  1. User

    Recommend me the best edge TPU or other portable solution to run a Whisper + 1.5b-parameters llm with the lower cost possible. Only provide solution that are in stock or avialable

  2. Perplexity Assistant

    For a cost-effective, portable solution to run Whisper and a 1.5B-parameter LLM, the Radxa Fogwise® AirBox is the top recommendation. Here’s why:

    Key Features

    • Performance:
      • Runs DeepSeek-R1-Distill-Qwen-1.5B at 30 tokens/s with INT4 quantization[3][5].
      • Compatible with Whisper for speech-to-text tasks[3][5].
    • Hardware:
      • 32 TOPS computational power with support for INT8/FP16/BF16/FP32 precision.
      • Up to 32GB RAM and expandable storage (supports 4TB via external drives)[3][5].
    • Cost Efficiency:
      • Open-source software stack and pre-optimized models reduce deployment costs.
      • Lower power consumption compared to GPU-based solutions[3][5].

    Availability

    • Currently in stock and openly documented on Radxa’s website[3][5].
    • Fully compatible with edge-optimized frameworks like TensorFlow Lite and PyTorch Mobile.

    Alternatives (If AirBox Unavailable)

    • Coral TPU (Edge TPU):
      • Limited to smaller models (e.g., Whisper-Lite) due to 8MB memory[7]. Not suitable for 1.5B LLMs.
    • Raspberry Pi 5 + eGPU:
      • Requires external GPUs (e.g., NVIDIA Jetson Orin Nano) for LLM inference, increasing cost and complexity[10].

    The Radxa AirBox provides the best balance of portability, performance, and cost for running both Whisper and a 1.5B LLM at the edge.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.