It's very usable for chat on a single Titan X maxwell, but it can't keep up at the rate I work at, that and the context windows don't seem to be large enough for my projects.
I use the crap out of it for other tasks. I'm currently playing with instruct models to write changelogs for me automatically in my CI system.
So far I've only used it heavily for writing research papers and stuff like that. I like that you can swap models and stuff though. Might build a serious rig this year to see what all is out there dev-wise.
I think that's on the list of all of us this year :)
The hardest thing for me is that I'm a mobile maxi. It gets tough to stay locked to a location. Which is why I've been rocking a laptop with peripherals for so many years now. I supposed I could probably remotely access it though... Lots to think about. Maybe a server rack is the way for me to go.
I hear you. My paranoia prevents me from having any persistent data on any of my client/workstation devices. Ollama + opewnwebui is the easiest way to do this IMO. You just get a really nice webui similar to chatgpt.com with way more features. It also ships with an authenticated OpenAI api you can use from curl if you want or other CLI clients.
