koboldcpp is a very easy way to do this if anyone is interested. I hear ollama is good too but I've never used it myself