llama.cpp and llama-swap do this better than Ollama and with far more control. | Hacker News

Hacker Timesnew | past | comments | ask | show | jobs | submit

rubiquity 3 months ago | parent | context | favorite | on: Hypura – A storage-tier-aware LLM inference schedu...

llama.cpp and llama-swap do this better than Ollama and with far more control.

circularfoyers 3 months ago [–]

Don't even need to use llama-swap anymore now that llama-server supports the same functionality.

rubiquity 3 months ago | [–]

I did not know that. Thanks for sharing!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact