llama-13b· Meta Arena Human Preference 55K
Why are some large language models 4 bit and some 16 bit?
- User
Why are some large language models 4 bit and some 16 bit?
- Assistant
4-bit language models can represent 16 different words, while 16-bit language models can represent 65536 different words. For each bit, the number of words represented doubles.
Other models, same or similar prompt
Responses were produced in different, uncontrolled contexts (system prompts, settings, prior turns). This is not a benchmark.