youre an llama.ccp with docke expoert the following works for me docker run -v…
- User
youre an llama.ccp with docke expoert the following works for me
docker run -v /mnt/g/data/llm/gemma-3-27b-it-Q4_K_M/:/models ghcr.io/ggerganov/llama.cpp:full --run -m /models/gemma-3-27b-it-Q4_K_M.gguf -p
now i have an gguf split in three files. How to modify the upper statement ?
Assistant
For split GGUF files with llama.cpp, you typically just need to point to the first part of the split model. Llama.cpp will automatically detect and load the other parts if they're in the same directory.
Assuming your split files follow a naming pattern like:
model.gguf.part1model.gguf.part2model.gguf.part3
Or:
model-00001-of-00003.ggufmodel-00002-of-00003.ggufmodel-00003-of-00003.gguf
You would modify your command to point to the first part:
docker run -v /mnt/g/data/llm/your-split-model/:/models ghcr.io/ggerganov/llama.cpp:full --run -m /models/model.gguf.part1 -pOr if using the numbered format:
docker run -v /mnt/g/data/llm/your-split-model/:/models ghcr.io/ggerganov/llama.cpp:full --run -m /models/model-00001-of-00003.gguf -pMake sure all the split parts are in the same directory that you're mounting (
/mnt/g/data/llm/your-split-model/in this example). Llama.cpp will automatically find and load the subsequent parts.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.