There’s a certain satisfaction to asking your self-hosted AI [tricky] questions and hear it turn on the fans as it computes.

Reply to this note

Please Login to reply.

Discussion

What hardware are you using, Ben?

Ryzen 5 7600, 30GB RAM, Nvidia RTX 3060, NVME M.2 with a hot swappable SSD RAID array, Noctua NF-A12 and NF-A8 fans in a Silverstone case that stacks in my hifi cabinet. Running Kubuntu with docker containers with Ollama, Open Web UI, Jellyfin. Will migrate lightning node to it soon. Will also swap out the graphics card at some point and improve airflow (side and back). Runs Deepseek-r1:14b comfortably (32b but slow response times) but would like to see if can run bigger models.

Swappable SSD RAID array🤤

OK, I haven’t tested that ;-)