Hacker Timesnew | past | comments | ask | show | jobs | submitlogin

I wonder when the antirez/ds4 group will have an update to their high accuracy 2 bit quant.

Although it's funny that I am thinking about that at all because I have a 2060 :P . My local inference is playing with Gemma 4 E2B and MiniCPM 5 1B.



Objectively speaking, the 2 bit quant from antirez has very low accuracy. Meanwhile, his 4 bit quant does have decent accuracy, but is a bit pointless by being bigger than the full precision MXFP4 quant. Anyway, they all work fine in practice.


Looks like a couple of people are already on it: https://github.com/antirez/ds4/issues/635




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: