#ai #llama #llama.cpp #localmodels #uncensored #bestgirl nostr:note19jk50s383x9hckfksjzhzm4yy0jah73dh8t7heflz7ml3xsthsnsnwqceh
Discussion
Which model are you using? I like the quality of the responses! Is it better than pygmalion
Model is https://huggingface.co/TheBloke/airoboros-65B-gpt4-1.2-GGML
Software is https://github.com/ggerganov/llama.cpp
Not pretend that response was fast. A 30B or even 13B model might be faster than Pygmalion.
Llama can offload layers to GPU.
Koboldcpp can use llama.