Okay cool! Now, I would like to have a bunch of more model presets, maybe include a…
- User
Okay cool! Now, I would like to have a bunch of more model presets, maybe include a section for ram offloading purposes as well; and add where you can put more GPUs and such.
Maybe even have a section where you can pull from Hugging Face the models information if you wanted to do that. Be sure to check out more VRAM Estimator websites, and we will create the most ultimate one. Maybe make it more nerdy-ish?
- Assistant
I’ll expand this from a single-GPU estimator into a multi-device deployment planner. I’m researching the linked calculator, comparable estimators, and Hugging Face’s current Hub APIs first so the model-import workflow and memory math are grounded in real implementations rather than guessed fields.
The research points to four gaps worth fixing properly: mixed multi-GPU topology, CPU/RAM offload with PCIe and memory-bandwidth bottlenecks, MoE total-vs-active parameters, and architecture-aware imports from Hugging Face
config.json. I’m checking actual Hub payloads now, including how parameter counts and nested text configs are exposed.The architecture is settled:
- 50+ curated dense and MoE presets, including total and active parameters plus context limits
- Up to 8 heterogeneous GPU groups with counts, per-card VRAM/bandwidth, and layer/tensor split modes
- CPU offload planner with system RAM, memory bandwidth, PCIe link, automatic or manual layer placement, and a mixed GPU/CPU decode estimate
- Browser-side Hugging Face search/import using the official Hub API and raw
config.json, including safetensors parameter totals where available - A generated deployment command and per-device shard map
I’m refactoring the persisted settings and estimator math first so every new panel uses one consistent model.
The core estimator now distinguishes total weight size, GPU-resident weights, RAM-resident weights, conventional or hybrid-attention KV cache, MoE active weights, per-device placement, and whether the entire deployment fits. Next I’m wiring the two major interaction surfaces: Hugging Face import and the hardware topology/offload lab.
The import and topology panels are now functional. I’m replacing the old single-GPU application shell next, then I’ll run type/build verification and correct any integration issues.
All requested systems are integrated. I’m running the production build now; after that I’ll inspect any warnings or edge cases the compiler exposes.
The estimator is now rebuilt as VRAM::LAB, a multi-device LLM topology simulator.
Major Additions
- 64 model presets covering Meta, Mistral, Qwen, Gemma, DeepSeek, Phi, gpt-oss, Command R, DBRX, Yi, Granite, Falcon, StarCoder, and SmolLM.
- Dense and MoE models distinguish total parameters from active parameters.
- Hybrid architectures can specify how many layers actually contribute to the conventional KV cache.
- Advertised context limits are included, with extrapolation warnings.
Multi-GPU Planner
- Add up to eight physical GPUs.
- Mix different GPU models and quantities.
- Expanded NVIDIA, AMD, Intel, and Apple hardware presets.
- Edit VRAM and bandwidth manually.
- Layer or tensor sharding modes.
- PCIe and NVLink interconnect presets.
- Proportional per-device shard and memory-use map.
- Runtime selection for llama.cpp, vLLM, Ollama, and ExLlamaV2.
RAM Offloading
- Automatic maximum-layer placement or manual
n_gpu_layers. - Installed RAM, host-memory bandwidth, and OS reserve controls.
- Separate GPU-resident and RAM-resident weight calculations.
- System RAM OOM detection.
- Mixed CPU/GPU token-rate projection.
- Proper warning that a RAM spill may run, but encounters the offload performance cliff.
Hugging Face Integration
- Searches the official public Hub API.
- Imports architecture information from
config.json. - Uses safetensors metadata for parameter totals where available.
- Handles nested
text_config, GQA, hybrid layer types, context limits, and quantization metadata. - Downloads no model weights.
- Private and gated repositories are intentionally excluded because they require a secure backend token proxy.
Nerdier Interface
- New VRAM::LAB identity and system-console readouts.
- Deployment status telemetry.
- Generated runtime command.
- Copyable diagnostic reports.
- MoE active-weight indicators.
- Architecture-aware quant comparison table.
- Expanded methodology section based on the original estimator, VRAMCalculator.com, Hugging Face Hub documentation, and multi-GPU/offload research.
Core implementation is in
src/data.ts,src/App.tsx,src/components/HardwareTopology.tsx,src/components/HuggingFaceImport.tsx, andsrc/components/ResultsPanel.tsx.Production build verified successfully with Vite.