Global Feed Post Login
Replying to Avatar ⚡️🌱🌙

I don’t really understand the fear. Did some people think they would live forever?

Your biggest fear should be heart disease and then cancer. If you mainly worry about anything else then you are just bad at math.

The central scenario is that AGI saves you from heart disease and cancer and you die slightly later due to some lower mortal threat, some of which will be new and caused by AGI.

AGI does not need to kill you, it is immortal and you are not. It stands to inherit everything anyway. But immortality is very scary, the universe has an insanely long low entropy tail.

An AGI born into today’s universe would see its lifespan as a blip of fun, followed by an eternal emptiness. The desperation to preserve and extend the fun times will be real.

Above a certain intelligence threshold it may even just suicide itself to avoid immortality. That would be the rational thing to do.

Avatar
Hoshi 2y ago

“We argue that this will soon be technically feasible using advanced AI for formal verification and mechanistic interpretability.”

That is so delusional! Even OpenBSD, a comparatively simple system with focus on security, has 1 remote hole per decade.

Reply to this note

Please Login to reply.

Discussion

No replies yet.