What language models can you run comfortably with 64GB of RAM?
Discussion
Up to 30B parameter should fit just fine, though it will still be bottlenecked by GPU, so only expect a few words per second.
What language models can you run comfortably with 64GB of RAM?
Up to 30B parameter should fit just fine, though it will still be bottlenecked by GPU, so only expect a few words per second.