Interestingly, some Manhattan project scientists were against the tests, finding them to be too early and dangerous for the whole world. They were worried about a 1:300,000 probability they missed something and the reaction caused by the explosion could burn the whole atmosphere.

The possibility of AGI going rouge and doing god knows what is much higher. It will be more intelligent than all of the humans combined after all, so we don’t know if it might decide that some megapolis is on the way of some more important cause.

I am not a fan of doom theories, but I believe we need tighter safety measures. Not sure how they should be put into place, but caution never harmed anyone so far AFAIK.