Avatar
Mariusz Kogen 1y ago

šŸš€ FreeGPT-2 v2.208.0310 is here!

Run #Llama 3.1, #NuExtract, #Mathstral, #GLM-4, #CodeGeeX, #Gemma 2 and many more models locally on StartOS. Powerful AI in your hands. No cloud needed.

https://marketplace.start9.com/free-gpt

#FreeGPT-2 on nostr:npub126ntw5mnermmj0znhjhgdk8lh2af72sm8qfzq48umdlnhaj9kuns3le9ll #StartOS

Reply to this note

Please Login to reply.

Discussion

Avatar
Wout32 1y ago

How much ram is needed Llama 3.1? Is 16gb enough?

Avatar
Mariusz Kogen 1y ago

Yes I have been running llama3.1:8b on Server One 2024 model with16GB Ram perfectly fine.

4f
Ollama 33 1y ago

Can it use the GPU?

Avatar
Mariusz Kogen 1y ago

Officially, this is CPU only, but unofficially, Iā€™m passing /dev/dri to that service, so Intel and AMD GPU acceleration theoretically should be possible.