Skip to content
TheChatArchiveThe Chat Archive
Claudeclaude-sonnet-4-20250514· AnthropicHugging Face LMArena Human Preference 140K

what options do i have for in client side browser llms

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with Metallama-3.3-70b-instruct’s answer.
  1. User

    what options do i have for in client side browser llms

  2. Claude Assistant

    There are several options for running LLMs directly in the browser:

    WebLLM is probably the most mature option. It uses WebGPU for acceleration and can run models like Llama 2, Vicuna, and others. It's developed by the MLC team and offers good performance for smaller models.

    Transformers.js by Hugging Face lets you run ONNX-converted models in the browser using WebAssembly. It supports a wide range of models but performance varies significantly based on model size.

    ONNX.js provides a more general framework for running ONNX models in browsers, which includes many language models that have been converted to the ONNX format.

    MediaPipe from Google has some text generation capabilities, though it's more focused on multimodal applications.

    Web-based implementations of smaller models like GPT-2 or DistilBERT are available through various JavaScript libraries.

    The main constraints you'll face are memory limitations (most browsers cap WebAssembly memory), computational speed (even with WebGPU, it's much slower than dedicated hardware), and model size (you're generally limited to smaller models under 1-2GB).

    WebLLM is probably your best starting point if you want something that works well out of the box. The performance will depend heavily on the user's hardware - newer devices with good GPUs will handle it much better than older or mobile devices.

    What's your intended use case? That might help narrow down the best option for your specific needs.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.