whats the best llm for an average person to replace something like chatgpt for daily…
- User
whats the best llm for an average person to replace something like chatgpt for daily tasks and life/universe questions
- Assistant
Here are a few strong contenders you can use “in place of” ChatGPT—whether you want something you can run completely locally, tap via API, or just out-of-the-box free. I’ve ordered them roughly by how “average-user friendly” they are:
Model Source How to Access Strengths Caveats OpenAI GPT-3.5-Turbo OpenAI (API) Sign up for free API credits Very capable generalist; low cost Requires internet & API key Llama 2 Chat (7B/13B) Meta (open source) Hugging Face, Docker images Local run; good “chat” fine-tuning Needs decent GPU (8–12 GB VRAM) Mistral Instruct Mix Mistral AI (open) Hugging Face, Replicate Excellent “instruction following” ~10 GB VRAM; a bit newer/unproven Anthropic Claude 2 Anthropic (API) Invite/API key Very safe/filtered; strong long-form Still “in closed beta” pricing Vicuna (7B) Community fine-tune Local via Hugging Face Superb chat quality for 7B footprint Large prompt overhead; tricky install —
Quick Recommendations
- If you just want “ChatGPT-like” and don’t mind API calls:
• GPT-3.5-Turbo. Free tier is generous; rock-solid generalist. - If you’d like total offline privacy (and have a capable PC):
• Llama 2 Chat 7B or 13B. Easy to spin up via Docker or a simple Python script. - If you love bleeding-edge open-source:
• Mistral Instruct Mix (if you can carve out the VRAM). It often edges out Llama 2 on instruction tasks.
—
What to consider
• Cost: API hits vs. free local inference
• Hardware: Do you have a GPU with ≥8 GB VRAM? If not, cloud/API is easier.
• Privacy: Local models never phone home; APIs do.
• Ease of use: OpenAI & Anthropic give you polished playgrounds. Open-source may need a bit of tinkering.Feel free to ask more about setup, pricing comparisons, or how to host these on your own machine! 😊
- If you just want “ChatGPT-like” and don’t mind API calls:
- User
i have 96gb of ram and a 5090
- Assistant
With your beefy 96 GB RAM + an RTX 5090, you can comfortably run large local models—up to ~70 B parameters (in 4-bit quant) or mid-sized ones in full precision. Below is a quick breakdown of top picks for “ChatGPT-style” daily chats, and roughly what you’ll need:
Model Approx. VRAM (fp16) Quantized VRAM (4-bit) Strengths Notes Mistral 7B ~14 GB ~7 GB SOTA small model—sharp instruction following, very snappy Great “first install” in text-generation-webui Llama 2 Chat 13B ~20 GB ~10 GB Official “chat” finetune by Meta; solid generalist Plug-and-play with Hugging Face recipes Falcon 40B ~48 GB ~24 GB Ultra-capable on reasoning + code tasks You’ll need the 4-bit GPTQ quant; 40B is heavy Llama 2 Chat 70B ~85 GB ~35 GB Very close to GPT-3.5-level quality 4-bit quant required + some CPU offload MPT-30B Instruct ~40 GB ~20 GB Excellent for “instruct” style prompts Open-source; has code & Q/A strengths —
Top 2 Recommendations
- Mistral 7B (4-bit)
• VRAM: ~7 GB (super light)
• Speed: ~200 tokens/sec on your 5090
• Why: Best “bang for buck” open-source chat model. Minimal fuss to get going—just grab the HF weights + text-generation-webui. - Llama 2 Chat 13B (4-bit)
• VRAM: ~10 GB
• Speed: ~120 tokens/sec
• Why: Officially supported “chat” finetune, very reliable on multi-turn convo and safe outputs.
—
If you want to push quality to the max
• Falcon 40B or Llama 2 70B in 4-bit: You’ve got the headroom. Expect ~24–35 GB VRAM usage, blazing results on deep Q&A, coding, brainstorming.
—
Getting Started
- Install text-generation-webui (or Hugging Face LLM runner).
- Download your chosen weights (e.g.
mistral-7b-instruct). - Launch in 4-bit mode with bitsandbytes:
python server.py \ --model mistral-7b-instruct \ --load-in-4bit \ --bnb-config bitsandbytes-config.json - Fire up the local playground at http://localhost:7860 🎉
Feel free to ask for a step-by-step setup guide, prompt-engineering tips, or how to squeeze even more performance out of your rig!
- Mistral 7B (4-bit)
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.