i have internalized the fact that i talk to my gpu on a daily basis and this is totally normal and not insane
Discussion
Or you could think of it as talking to the ghost of all scrapeable content on the internet, which is also really weird
I’m a calm person, I almost never talk harshly to a human or animal. I insult inanimate objects frequently though😂
I have conversations with my cats. To be fair, they do meow back. Sometimes we actually do understand each other (in super limited ways, obviously). It's kind of weird to have that with an animal. Never had it with a GPU though. GPUs are probably the future girlfriend for some people anyway. 😂
GPU addiction is real :-(😀
https://www.reddit.com/r/ChatGPT/comments/zghxcx/chatgpt_addiction/?rdt=41082
Which model? So far llama3.3 is my only tolerable local model. But it is throttled by the speed my ram can feed the remaining 18GB to my CPU. So mostly I talk to my CPU I guess even though the GPU is doing 4/7 of the work.
I get about 3.5 tokens/sec. A 5090 is tempting simply because it would cut RAM bound performance in half.
Yeah llama is what i use. Its fast
At 70B parameters? 3.3 t/s is equivalent to a fairly fast human typist, but not so fast I don't pick and choose what to ask it. Works pretty well with Continue AI. But the default context window in ollama is kinda small if I need it to look at more than a few files.
"You will not always have a calculator in your pocket"
...except really you probably will, or someone near you will, and it'll understand & speak english or whatever language you prefer 🤷♂️