what is the best LLM for local coding under 20b parameters when I have 16 GB of VRAM?
- User
what is the best LLM for local coding under 20b parameters when I have 16 GB of VRAM?
- Assistant
Given your 16 GB of VRAM and the requirement for a local coding-focused language model under 20 billion parameters, here are some suitable options:
Code Llama 13B
A 13-billion parameter model fine-tuned for code generation tasks, offering state-of-the-art performance among open models[1].StarCoder 15.5B
A 15.5-billion parameter model trained on a diverse set of programming languages, supporting an 8K context length and infilling capabilities[2].Granite Code Models 13B
A 13-billion parameter model optimized for enterprise software development workflows, performing well across a range of coding tasks[3].GPT-NeoX-20B
A 20-billion parameter autoregressive language model trained on a diverse dataset, offering strong few-shot reasoning capabilities[4].Among these, Code Llama 13B is particularly noteworthy for its specialized training on code-related tasks, making it a strong candidate for your coding needs. However, given your VRAM constraints, it's advisable to use quantization techniques to reduce memory usage and ensure smooth operation. Tools like llama.cpp can assist in running these models efficiently on hardware with limited resources.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.