> Is Python so annoying to use that when a compelling non-Python solution appears, everyone will love it? Less hassle? Or did it take off for a different reason?
The desktop/laptop LLMs use case is one where resource efficiency often makes the difference between “I can’t use this at all” rather than the more frequent “I can use it but it maybe runs a little slower”, and llama.cpp offers that. It also has offered new quantization options that the Python-based tooling hasn’t, which compounds the basic resource efficiency point.
(It’s also not an “everyone” thing: plenty of people are using the Python-based toolchains for LLMs, its possible for their to be multiple popular options in a space . Not everything is all or nothing.)
The desktop/laptop LLMs use case is one where resource efficiency often makes the difference between “I can’t use this at all” rather than the more frequent “I can use it but it maybe runs a little slower”, and llama.cpp offers that. It also has offered new quantization options that the Python-based tooling hasn’t, which compounds the basic resource efficiency point.
(It’s also not an “everyone” thing: plenty of people are using the Python-based toolchains for LLMs, its possible for their to be multiple popular options in a space . Not everything is all or nothing.)