Nostr Web Client

Is a gguf the big model file that ends with .safetensors? Sorry I am new to this

No, sorry, that’s the file extension. For example, this page has some large *.gguf files split into parts (because Hugging Face has a max upload size of 50GB): https://huggingface.co/mradermacher/dolphin-2.9.2-mixtral-8x22b-GGUF

Once you download the two parts, you can combine them into the single *.gguf file that ollama is able to import. Instructions for combining the part files can be found here: https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF

Reply to this note

Please Login to reply.

Discussion

Hypnagog 1y ago

Well I guess thats fine for models with gguf files.... But ive never seen those. Take this model for example (satoshi), there's no gguf just safetensors files

https://huggingface.co/LaierTwoLabsInc/Satoshi-7B/tree/main

jimbocoin 🃏 1y ago

Yeah, I believe there are tools that can covert them but I haven’t tried. Once I found that there were already gguf files for the models I wanted to run, I just used those.

If you try the conversion tools, let me know how it goes!