this post was submitted on 26 Jul 2023
19 points (100.0% liked)

LocalLLaMA

2249 readers
1 users here now

Community to discuss about LLaMA, the large language model created by Meta AI.

This is intended to be a replacement for r/LocalLLaMA on Reddit.

founded 1 year ago
MODERATORS
pax@sh.itjust.works
SkySyrup@sh.itjust.works
noneabove1182@sh.itjust.works
19
What is better: higher quantiation or higher parameter count? (yiffit.net)
submitted 1 year ago by Wander@yiffit.net to c/localllama@sh.itjust.works
16 comments fedilink hide all child comments
 

For example, does a 13B parameter model at 2_K quantiation perform worse than a 7B parameter model at 8bit or 16bit?

you are viewing a single comment's thread
view the rest of the comments
[–] noneabove1182@sh.itjust.works 2 points 1 year ago

These are good sources, to add one more, the GPTQ paper talks a lot about perplexity at several quantization and model sizes:

https://arxiv.org/abs/2210.17323