The really sucky thing about using AI is that you can’t verify what is giving you responses. You cannot trust it.

They can tell you the base model, but you have to believe them; and even if they could prove it, most #LLM providers are running orchestration layers on top of the #AI feeding it other data and allowing the AI to fetch its own (by proxy) external data. So there’s no telling where it builds its response from.

The safest would be to self host. You cannot trust it verify the base models you have. And then you can run your own tooling/orchestration on the model.

For now we don’t have better solution, so we just need to be cautious.

But that seems like the only sane and safe future for me.

Let’s make it happen

Reply to this note

Please Login to reply.

Discussion

Self-hosting has been working great for me. Qwen3 32B Q6 meets most of my general needs.

I only have 12G so a 32B is not possible even with quantization. So I mostly use 7B/8B models

Works for me too.

What are you using as an interface to yours? LMStudio, Ollama, Gpt4all?

I'm fortunate to have ~40 (16+24). I use oobabooga text-generation-webui.

Ah I’ve used that before.

Wow. A 16 and 24. Sounds heavenly.

Will need to buy another sometime for sure. I was dumb to go for a 12 instead of 16 when I got my card.