If you're already familiar with Linux in general, this repo will tell you what to do.

To get the actual models, lookup "TheBloke" on Hugging Face.

https://github.com/ggerganov/llama.cpp