No for the text generation inference (the server that runs the llms), I was researching the different options available, some cool ones are huggingface tgi, llama.cpp, vllm
No for the text generation inference (the server that runs the llms), I was researching the different options available, some cool ones are huggingface tgi, llama.cpp, vllm
No replies yet.