Replying to Avatar aljaz

nostr:nprofile1qyfhwumn8ghj7ctvvahjuat50phjummwv5q32amnwvaz7tm9v3jkutnwdaehgu3wd3skueqqyzu7we2xhgry2mknq8v7227yn7jguu9xhu3g90n6rtnjj3mpyq3ackdvvhl i've seen reports that theoretically you can get up to 20t/s with this setup, practically might be less so there might be better ways to spend 10k to get more juice out of the hw (tho smaller model)

I would probably prioritize the larger models, 20t/sec seems fine, especially for what I’d be more likely to use it for (Whisper, Florence, Hunyuan, Stable Diffusion) where the LLM is mostly a go between and/or “organizer.”

But I’d probably also change how I use most of my Ai tools if I could run the largest models, so it might suddenly become something I would notice because I changed how I was using things.

Reply to this note

Please Login to reply.

Discussion

I'm gonna wait for nvidia digits first, while much smaller vram if the original price point will be roughly on point you could get 128gb vram for around $3k (obviously it will be more) but i'm curious to see how price/performance will compare, specially if you could get multiple Digits for the same amount

Yeah I was thinking hard about that one too. But I’m not sure they were going to offer the 128GB at the $3K price. If they actually are then that’s awesome, I’ll get 3 of the damn things, lol. But the wording made me think “oh it STARTS at $3K”

yeah unlikely. but i also want blackwell for other reasons, gives you full tee in a gpu 🥳

also there's one crucial difference too, i think its gonna be impossible to get your hands on any nvidia hardware when they release it, like try getting jetson anywhere

full screenshot with asus version as well

That’s funny I was just jumping back here directly from reserving a double of them 🤣

I’m not 100% certain I’ll get them, but we will see how well I’ve integrated and can utilize everything when it gets here.

haha I reserved a GX10 tho tbh I didn't even find much info about it besides the storage diff

lets see if i'm on the right continent to get it when they become available