Koboldcpp is a good llm front-end that I use. The github had pretty good instructions for how to install. You then get an llm from huggingface (I use mixtral) you just run koboldcpp with the path to your model as an argument and open the web front-end in your browser.