very cool setup. Did you try running Mistral 7b?
There’s still some final touches but this thing has been reliably crunching numbers for 3 days now. This thing is solid 🔥🔥
Running Prem Ai with Vicuna 3b, Whisper, and Stable Diffusion. All are cruising nicely. So cool having my own Ai to work with. Exploring some coding specific models tomorrow. https://v.nostr.build/azoQ.mov