Hacker News
new
|
past
|
comments
|
ask
|
show
|
jobs
|
submit
login
cbo100
on Jan 21, 2025
|
parent
|
context
|
favorite
| on:
DeepSeek-R1
I get the right answer on the 8B model too.
It could be the quantized version failing?
ein0p
on Jan 21, 2025
[–]
My models are both 4 bit. But yeah, that could be - small models are much worse at tolerating quantization. That's why people use LoRA to recover the accuracy somewhat even if they don't need domain adaptation.
Consider applying for YC's Fall 2026 batch!
Applications
are open till July 27.
Guidelines
|
FAQ
|
Lists
|
API
|
Security
|
Legal
|
Apply to YC
|
Contact
Search:
It could be the quantized version failing?