What is the state of the art self hosted AI for conversation? Anything decent?
Discussion
I haven't tried myself yet... but it seems that Llama2 is the "current thing" for self hosted AI, I enjoyed this video showing how to install everything locally: https://www.youtube.com/watch?v=k2FHUP0krqg
Saw the video and found it very interesting although still needs to be more reliable. I wonder if with more tokens gets more accurate. Will check more of his vídeos. Thanks!
Maybe nostr:npub1l2vyh47mk2p0qlsku7hg0vn29faehy9hy34ygaclpn66ukqp3afqutajft and nostr:npub1klkk3vrzme455yh9rl2jshq7rc8dpegj3ndf82c3ks2sk40dxt7qulx3vt have more info about this?
Asking for models or tools?
For tools check out secondbrain.sh, serge.chat, faraday.dev.
For models Wizard-Vicuna-Uncensored is very good at question answering and ehartford/based gives amusing conversations.
To me this is the most interesting llama related project right now:
https://github.com/ggerganov/llama.cpp
Very easy to setup:
1. git clone the project and setup it
https://github.com/ggerganov/llama.cpp#get-the-code
2. Select your LLaMA2 model already quantizated from here: https://huggingface.co/TheBloke/Llama-2-7B-Chat-GGML/tree/main
3. ./main -m ./models/your_downloaded_model.bin -n 128
Have to test it. Thank you Desobediente. Question, do you know how much space it uses?