Skip to content
TheChatArchiveThe Chat Archive
Claudeclaude-opus-4-20250514-thinking-16k· AnthropicHugging Face LMArena Human Preference 140K

youre an llama.ccp with docke expoert the following works for me docker run -v…

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with Googlegemma-3-27b-it’s answer.
  1. User

    youre an llama.ccp with docke expoert the following works for me

    docker run -v /mnt/g/data/llm/gemma-3-27b-it-Q4_K_M/:/models ghcr.io/ggerganov/llama.cpp:full --run -m /models/gemma-3-27b-it-Q4_K_M.gguf -p

    now i have an gguf split in three files. How to modify the upper statement ?

  2. Claude Assistant

    For split GGUF files with llama.cpp, you typically just need to point to the first part of the split model. Llama.cpp will automatically detect and load the other parts if they're in the same directory.

    Assuming your split files follow a naming pattern like:

    • model.gguf.part1
    • model.gguf.part2
    • model.gguf.part3

    Or:

    • model-00001-of-00003.gguf
    • model-00002-of-00003.gguf
    • model-00003-of-00003.gguf

    You would modify your command to point to the first part:

    docker run -v /mnt/g/data/llm/your-split-model/:/models ghcr.io/ggerganov/llama.cpp:full --run -m /models/model.gguf.part1 -p
    

    Or if using the numbered format:

    docker run -v /mnt/g/data/llm/your-split-model/:/models ghcr.io/ggerganov/llama.cpp:full --run -m /models/model-00001-of-00003.gguf -p
    

    Make sure all the split parts are in the same directory that you're mounting (/mnt/g/data/llm/your-split-model/ in this example). Llama.cpp will automatically find and load the subsequent parts.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.