Skip to content
TheChatArchiveThe Chat Archive
Perplexitysonar-reasoning· PerplexityHugging Face LMArena Search Arena 24K

python ./koboldcpp.py *** Welcome to KoboldCpp - Version 1.86.2 For command line…

From a Chatbot Arena battleThe same person sent this prompt to two anonymous models at once. Compare it with gpt-4o-mini-search-preview’s answer.
  1. User

    python ./koboldcpp.py


    Welcome to KoboldCpp - Version 1.86.2 For command line arguments, please refer to --help


    Unable to detect VRAM, please set layers manually. Auto Selected Default Backend...

    Loading Chat Completions Adapter: /Users/mac/Downloads/koboldcpp/kcpp_adapters/AutoGuess.json Chat Completions Adapter Loaded Initializing dynamic library: koboldcpp_vulkan.so

    ImageGen Init - Load Model: /Users/mac/Downloads/models/LCM_Dreamshaper_v7-q5_0.gguf

    MODEL:/Users/mac/Downloads/models/LCM_Dreamshaper_v7-q5_0.gguf VAE: TAESD: CNET: LORA: EMBD: VAE_DEC:0 VAE_TILE:0 FREE_PARAM:0 THREADS:2 WTYPE:39 RNGTYPE:1 SCHED:0 CNETCPU:0

    Using Metal backendggml_metal_init: allocating ggml_metal_init: picking default device: Intel HD Graphics 5000 ggml_metal_load_library: default.metallib not found, loading from source ggml_metal_load_library: GGML_METAL_PATH_RESOURCES = /Users/mac/Downloads/koboldcpp ggml_metal_load_library: loading '/Users/mac/Downloads/koboldcpp/ggml-metal-merged.metal' ggml_metal_init: GPU name: Intel HD Graphics 5000 ggml_metal_init: GPU family: MTLGPUFamilyCommon2 (3002) ggml_metal_init: simdgroup reduction = false ggml_metal_init: simdgroup matrix mul. = false ggml_metal_init: has residency sets = false ggml_metal_init: has bfloat = false ggml_metal_init: use bfloat = false ggml_metal_init: hasUnifiedMemory = true ggml_metal_init: recommendedMaxWorkingSetSize = 1610.61 MB ggml_metal_init: loaded kernel_add 0x7fb6ccf4b0e0 | th_max = 1024 | th_width = 16 ggml_metal_init: loaded kernel_add_row 0x7fb6ccf62390 | th_max = 1024 | th_width = 32 ggml_metal_init: loaded kernel_sub 0x7fb6ccf630f0 | th_max = 1024 | th_width = 16 ggml_metal_init: loaded kernel_sub_row 0x7fb6ccf60c00 | th_max = 1024 | th_width = 32 ggml_metal_init: loaded kernel_mul 0x7fb6ccf7a3e0 | th_max = 1024 | th_width = 16 ggml_metal_init: loaded kernel_mul_row 0x7fb6ccfa1570 | th_max = 1024 | th_width = 32 ggml_metal_init: loaded kernel_div 0x7fb6ccfa21c0 | th_max = 1024 | th_width = 16 ggml_metal_init: loaded kernel_div_row 0x7fb6ccfa2e10 | th_max = 1024 | th_width = 32 ggml_metal_init: loaded kernel_repeat_f32 0x7fb6ccfa3a60 | th_max = 1024 | th_width = 16 ggml_metal_init: loaded kernel_repeat_f16 0x7fb6ccfa46b0 | th_max = 1024 | th_width = 16 ggml_metal_init: loaded kernel_repeat_i32 0x7fb6ccfa5300 | th_max = 1024 | th_width = 16 ggml_metal_init: loaded kernel_repeat_i16 0x7fb6ccfa5f50 | th_max = 1024 | th_width = 16 ggml_metal_init: loaded kernel_scale 0x7fb6ccfa6c80 | th_max = 1024 | th_width = 32 ggml_metal_init: loaded kernel_scale_4 0x7fb6ccfa7850 | th_max = 1024 | th_width = 32 ggml_metal_init: loaded kernel_clamp 0x7fb6ccfa84a0 | th_max = 1024 | th_width = 32 ggml_metal_init: loaded kernel_tanh 0x7fb6ccfa90f0 | th_max = 1024 | th_width = 32 ggml_metal_init: loaded kernel_relu 0x7fb6ccfa9d40 | th_max = 1024 | th_width = 32 ggml_metal_init: loaded kernel_sigmoid 0x7fb6ccfaa990 | th_max = 1024 | th_width = 32 ggml_metal_init: loaded kernel_gelu 0x7fb6ccfab5e0 | th_max = 1024 | th_width = 32 ggml_metal_init: loaded kernel_gelu_4 0x7fb6ccfac230 | th_max = 1024 | th_width = 32 ggml_metal_init: loaded kernel_gelu_quick 0x7fb6ccface80 | th_max = 1024 | th_width = 32 ggml_metal_init: loaded kernel_gelu_quick_4 0x7fb6ccfadad0 | th_max = 1024 | th_width = 32 ggml_metal_init: loaded kernel_silu 0x7fb6ccfae720 | th_max = 1024 | th_width = 32 ggml_metal_init: loaded kernel_silu_4 0x7fb6ccfaf370 | th_max = 1024 | th_width = 32 ggml_metal_init: loaded kernel_elu 0x7fb6ccfb00d0 | th_max = 1024 | th_width = 32 ggml_metal_init: skipping kernel_soft_max_f16 (not supported) ggml_metal_init: skipping kernel_soft_max_f16_4 (not supported) ggml_metal_init: skipping kernel_soft_max_f32 (not supported) ggml_metal_init: skipping kernel_soft_max_f32_4 (not supported) ggml_metal_init: loaded kernel_diag_mask_inf 0x7fb6c9618f30 | th_max = 1024 | th_width = 32 ggml_metal_init: loaded kernel_diag_mask_inf_8 0x7fb6c9651e40 | th_max = 1024 | th_width = 32 ggml_metal_init: loaded kernel_get_rows_f32 0x7fb6ccfb11d0 | th_max = 1024 | th_width = 16 ggml_metal_init: loaded kernel_get_rows_f16 0x7fb6ccfb1e70 | th_max = 1024 | th_width = 16 ggml_metal_init: skipping kernel_get_rows_bf16 (not supported) ggml_metal_init: loaded kernel_get_rows_q4_0 0x7fb6c9582e20 | th_max = 1024 | th_width = 32 ggml_metal_init: loaded kernel_get_rows_q4_1 0x7fb6c954f590 | th_max = 1024 | th_width = 32 ggml_metal_init: loaded kernel_get_rows_q5_0 0x7fb6c95a4020 | th_max = 1024 | th_width = 32 ggml_metal_init: loaded kernel_get_rows_q5_1 0x7fb6c9593740 | th_max = 1024 | th_width = 32 ggml_metal_init: loaded kernel_get_rows_q8_0 0x7fb6c959e430 | th_max = 1024 | th_width = 32 ggml_metal_init: loaded kernel_get_rows_q2_K 0x7fb6c962ead0 | th_max = 1024 | th_width = 32 ggml_metal_init: loaded kernel_get_rows_q3_K 0x7fb6c9639140 | th_max = 1024 | th_width = 32 ggml_metal_init: loaded kernel_get_rows_q4_K 0x7fb6cce94400 | th_max = 1024 | th_width = 32 ggml_metal_init: loaded kernel_get_rows_q5_K 0x7fb6cce92f20 | th_max = 1024 | th_width = 32 ggml_metal_init: loaded kernel_get_rows_q6_K 0x7fb6ccfb1140 | th_max = 1024 | th_width = 32 ggml_metal_init: loaded kernel_get_rows_iq2_xxs 0x7fb6ccfb2800 | th_max = 1024 | th_width = 32 ggml_metal_init: loaded kernel_get_rows_iq2_xs 0x7fb6cce929d0 | th_max = 1024 | th_width = 32 ggml_metal_init: loaded kernel_get_rows_iq3_xxs 0x7fb6cce51d50 | th_max = 1024 | th_width = 32 ggml_metal_init: loaded kernel_get_rows_iq3_s 0x7fb6ccfb4440 | th_max = 1024 | th_width = 32 ggml_metal_init: loaded kernel_get_rows_iq2_s 0x7fb6ccfb5120 | th_max = 1024 | th_width = 32 ggml_metal_init: loaded kernel_get_rows_iq1_s 0x7fb6c9646db0 | th_max = 1024 | th_width = 32 ggml_metal_init: loaded kernel_get_rows_iq1_m 0x7fb6ccfb6000 | th_max = 1024 | th_width = 32 ggml_metal_init: loaded kernel_get_rows_iq4_nl 0x7fb6ccfb6dc0 | th_max = 1024 | th_width = 32 ggml_metal_init: loaded kernel_get_rows_iq4_xs 0x7fb6ccfb7c80 | th_max = 1024 | th_width = 32 ggml_metal_init: loaded kernel_get_rows_i32 0x7fb6ccfb8b00 | th_max = 1024 | th_width = 16 ggml_metal_init: skipping kernel_rms_norm (not supported) ggml_metal_init: skipping kernel_group_norm (not supported) ggml_metal_init: loaded kernel_norm 0x0 | th_max = 0 | th_width = 0 ggml_metal_init: error: load pipeline error: Error Domain=CompilerError Code=2 "Compiler encountered an internal error" UserInfo={NSLocalizedDescription=Compiler encountered an internal error} ggml_backend_metal_init: error: failed to allocate context

    Using Vulkan backendggml_vulkan: Found 1 Vulkan devices: ggml_vulkan: 0 = Intel HD Graphics 5000 (MoltenVK) | uma: 1 | fp16: 1 | warp size: 32 | shared memory: 32768 | matrix cores: none

    loading model from '/Users/mac/Downloads/models/LCM_Dreamshaper_v7-q5_0.gguf' load /Users/mac/Downloads/models/LCM_Dreamshaper_v7-q5_0.gguf using gguf format init from '/Users/mac/Downloads/models/LCM_Dreamshaper_v7-q5_0.gguf' Version: SD 1.x Weight type: q5_0 Conditioner weight type: q5_0 Diffusion model weight type: q5_0 VAE weight type: q5_0 ggml tensor size = 400 bytes vocab size: 49408 trigger word img already in vocab clip params backend buffer size = 81.25 MB(VRAM) (196 tensors) unet params backend buffer size = 1304.80 MB(VRAM) (686 tensors) vae params backend buffer size = 159.68 MB(VRAM) (248 tensors) loading weights loading tensors from /Users/mac/Downloads/models/LCM_Dreamshaper_v7-q5_0.gguf total params memory size = 1545.73MB (VRAM 1545.73MB, RAM 0.00MB): clip 81.25MB(VRAM), unet 1304.80MB(VRAM), vae 159.68MB(VRAM), controlnet 0.00MB(VRAM), pmid 0.00MB(VRAM) loading model from '/Users/mac/Downloads/models/LCM_Dreamshaper_v7-q5_0.gguf' completed, taking 8.08s running in eps-prediction mode finished loaded fileLoad Image Model OK: True Embedded KoboldAI Lite loaded. Embedded API docs loaded. Embedded SDUI loaded.

    Active Modules: ImageGeneration Inactive Modules: TextGeneration VoiceRecognition MultimodalVision NetworkMultiplayer ApiKeyPassword [PASSWORD] TextToSpeech AdminControl Enabled APIs: KoboldCppApi A1111ForgeApi ComfyUiApi Starting Kobold API on port 5001 at http://localhost:5001/api/ Starting OpenAI Compatible API on port 5001 at http://localhost:5001/v1/ StableUI is available at http://localhost:5001/sdui/

    Please connect to custom endpoint at http://localhost:5001 env: xdg-open: No such file or directory

    Input: {"steps": 4, "n": 1, "sampler_name": "Euler", "width": 512, "height": 512, "cfg_scale": 1, "seed": -1, "denoising_strength": 0.1, "prompt": "mountain road", "negative_prompt": "", "init_images": []}

    [13:23:35] Generating Image (4 steps)

    TXT2IMG PROMPT:mountain road NPROMPT: CLPSKP:-1 CFGSCLE:1.000000 W:512 H:512 SM:1 STEP:4 SEED:-1 BATCH:1 CIMG:0x0 CSTR:0.900000

    txt2img 512x512 prompt after extract and remove lora: "mountain road" apply_loras completed, taking 0.00s parse 'mountain road' to [['mountain road', 1], ] token length: 77 clip compute buffer size: 1.40 MB(VRAM) computing condition graph completed, taking 1077 ms get_learned_condition completed, taking 1085 ms sampling using Euler method generating image: 1/1 - seed 1272957719 unet compute buffer size: 559.90 MB(VRAM)[mvk-error] VK_ERROR_INITIALIZATION_FAILED: Shader library compile failed (Error code 3): Compilation failed:

    program_source:129:32: error: reinterpret_cast from 'const device _66 *const constant' to 'const device _180 *const constant &' is not allowed const device auto &_182 = reinterpret_cast<const device _180 constant &>(spvDescriptorSet0.m_68); ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ . [mvk-error] VK_ERROR_INVALID_SHADER_NV: Compute shader function could not be compiled into pipeline. See previous logged error. ggml_vulkan: Compute pipeline creation failed for mul_mat_vec_q5_0_f32_f32_1 ggml_vulkan: vk::Device::createComputePipeline: ErrorInvalidShaderNV libc++abi: terminating with uncaught exception of type std::out_of_range: unordered_map::at: key not found zsh: abort python ./koboldcpp.py

  2. Perplexity Assistant

    The error you're encountering when running KoboldCpp with the LCM_Dreamshaper_v7-q5_0.gguf model appears to stem from shader compilation failures and hardware compatibility issues with your Intel HD Graphics 5000 GPU. Here's a breakdown of the key points:


    Error Analysis

    1. Metal Backend Failure
      The Metal backend initialization fails with a compiler error (Compiler encountered an internal error), likely due to unsupported GPU features or outdated drivers[9]. Intel HD Graphics 5000 is an older GPU, which may lack required Metal capabilities for modern LLM workloads.

    2. Vulkan Backend Failure
      The Vulkan backend fails with a shader compilation error (reinterpret_cast issue), indicating incompatibility with the GPU's architecture or driver limitations[9]. The error message suggests the shader code generated by KoboldCpp isn't compatible with your GPU.

    3. Model Quantization (q5_0)
      The model uses q5_0 quantization, which may require specific GPU support for mixed-precision operations. Older GPUs like the Intel HD 5000 may not handle these operations efficiently or at all.


    Potential Solutions

    1. Hardware Limitations

    • Upgrade GPU: Intel HD Graphics 5000 is not suitable for modern LLM workloads. Consider using a dedicated GPU (e.g., NVIDIA RTX 20/30/40 series) for better compatibility with CUDA/Vulkan backends[3][6].
    • Cloud Services: Use services like Novita AI for GPU-accelerated KoboldCpp instances (e.g., NVIDIA RTX 4090)[6].

    2. Backend Configuration

    • Force CPU Mode: If GPU acceleration isn’t critical, disable GPU offloading entirely.
    • Adjust GPU Layers: Manually reduce the number of GPU layers to avoid VRAM overflows, though this may not resolve shader issues[3].

    3. Model Compatibility

    • Use Smaller Models: Try models with lower quantization (e.g., q4_0) or smaller architectures (e.g., 7B instead of 13B/16B)[5].
    • Check Model Integrity: Ensure the model file (LCM_Dreamshaper_v7-q5_0.gguf) is undamaged and compatible with KoboldCpp’s GGUF format[3].

    Troubleshooting Steps

    1. Verify GPU Support
      Confirm your GPU supports Metal/Vulkan and the required compute capabilities.
    2. Update Drivers
      Ensure Intel GPU drivers are up-to-date (though this may not resolve shader issues).
    3. Test with a Simpler Model
      Load a smaller model (e.g., SDXL-c210-Draw) to isolate whether the issue is model-specific or hardware-related[3].

    If hardware upgrades aren’t feasible, consider using cloud-based solutions or alternative LLM frameworks optimized for older CPUs/GPUs.

Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.