Yeah, there are lots of potential threats of AGI that are silly and some of them are more realistic. Also, some are not very bad and some are really bad I would say.

To put what I’m thinking about more concretely, Elon Musk’s goal for Mars is that if the ships ever stop coming from earth, they will be fine. What does that require… Self-sufficiency. Independence from earth (and humans here) for power, data, raw materials, etc.

So the comparison to the Martian colony, I think should clarify that I’m not imagining a Godlike magical entity at all, obviously humans can do exactly the same thing: create a fully self sufficient hive that is separable from earth. If Musk gets his way that will be a reality this century.

AI will be a big part in making that happen, but if it gets to 100% then the hypothetical science fiction sounding threat becomes possible.

And that is to say nothing about one of these things, becoming smart enough to set up in the meantime a system like what Aldous Huxley described “The perfect dictatorship would have the appearance of a democracy but would basically be a prison without walls in which the prisoners would not even dream of escaping “