but everything the generative AI models spit out right now is pretty wrong and inaccurate except for simple to moderate use cases. I'm one of those people who only asks StackOverlfow when I'm really stuck, and the response ChatGPT has given me has almost always been wrong (or partially interesting but still mostly wrong) ...

They can string together things for solved problems, but it's not great at new or unvisited problems .. and we don't really understand how they work. It's just massive tables of back-propagated weights and values that get us from input to output in large language models.

I think the bigger problem is the general population putting confidence is broken models, and how much will be deferred to in the future by "Well the AI [gods] told us so" ... it's like God 2.0, but more of a regression .. so like God 0.3 ... obey your AI God when it tells you to sacrifice your children