Replying to alphakamp

In my experiments, I am painfully frustrated with the hardware on hand for AI. Ollama, LMstudio, Jan, gpt4all. Let's say 30 tokens max with 8b. Tried out some basic local docs stuff with a single PDF/CSV, most 8b models can even read a basic spreadsheet without failing miserably
cc
alphakamp 11mo ago

Even with two columns and simple rows, deepseek distill 8b is literally retarded. Is this because of my ouny 4gb vram? This shit is lame

Reply to this note

Please Login to reply.

Discussion

No replies yet.