Global Feed Post Login
Replying to Avatar John

24 gb VRAM (3090, 7900 cards): the latest mistral 24b, qwen3 32b and qwen3 30a3 (MoE)

48gb: 70b size models at decent quants, mistral dev large at lobotomized quants. Mistral dev large is the main one in this bracket. There might be other good 70b's released lately

96gb: gpt-oss 120b

This is to fit everything in vram. With MoE's (qwen3 30a3, GPT-oss) you can get by with VRAM+RAM without ruining your speed depending on the speed of your ram.

But it's usually a speed hit so I don't use anything that doesn't fit in VRAM

Avatar
Kajoozie Maflingo 2w ago

yup, and about 2tb storage just to fit all those models.

Reply to this note

Please Login to reply.

Discussion

No replies yet.