Can it use the GPU?
š FreeGPT-2 v2.208.0310 is here!
Run #Llama 3.1, #NuExtract, #Mathstral, #GLM-4, #CodeGeeX, #Gemma 2 and many more models locally on StartOS. Powerful AI in your hands. No cloud needed.
https://marketplace.start9.com/free-gpt
#FreeGPT-2 on nostr:npub126ntw5mnermmj0znhjhgdk8lh2af72sm8qfzq48umdlnhaj9kuns3le9ll #StartOS
Discussion
Officially, this is CPU only, but unofficially, Iām passing /dev/dri to that service, so Intel and AMD GPU acceleration theoretically should be possible.