If you wanted to build/buy a machine expressly for the purposes of running private AI agents for general work (transcription, summarizing, text to speech, coding, chat, etc...), what would you build/buy?
Discussion
If power cost is not a big issue, I'd buy a used enterprise workstation with 1 or 2 CPUs, and 256+ GB ram, then find two 24GB+ Nvidia GPUs that will fit (remember to consider power requirements wrt PSU). RTX 3090's are pretty popular afaik
My understanding of the AMD Ryzen AI chip that they are going to use in the new Framework Desktop is that the onboard memory can be allocated to the GPU allowing for any sized model to run. Framework is in pre-order, but I have to imagine others built with this chip are out or soon to be.