This is good info, thanks. I want to do something similar, but know very little about how to set the components of LLMs up.

I've read a bit on what the various components are. What I don't see in your comment is what you're using to run your model locally. Ollama?