Replying to Avatar ElectronicsQuestions

Reply to #[1] about the episode mentioned here. Probably an unpopular opinion, since I don't have, and can't have, a solution, except for maybe going 100% isolated off-grid farmer/prepper/survivalist small village, refusing to believe any outside information whatsoever, or to some extent prevention by teaching AI Austrian economics, but:

I believe even your take, and indeed my following take, severely underestimates the damage AI can cause if it goes wrong. What if it's not 1% worse per minute of whatever the AI is used for, but 1% per microsecond, and it also figures out a way to spread the -1% into all adjacent systems? If it's figuratively not turning the frog-boiling stove to maximum, but figures out a way to do thermonuclear fusion of all hydrogen atoms in the water in the pot, in less than the time it takes for the first nerve signal to reach the frog's brain? Exponential developments is something we're not naturally able to intuit. Even less exponential exponents, which I imagine could possibly come into play with AI at some point.

If I recall correctly, public key cryptography is related to the P/NP problem, and there is no way of mathematically proving this. That leads me to believe that there is a far from zero probability that P=NP, and if AI develops exponentially, it will definitely find how. One could argue that AI could also solve this by creating better encryption schemes, but if an attacking AI figures it out one second earlier, the entire Web of Trust is gone, and if it figures it out a number of blocks earlier - which it could possibly solve in fractions of seconds too, thanks to the discovery - all 21 million bitcoin is already in its wallet. Billions of people would probably die within days to weeks, from hunger, thirst or not being able to heat their homes.

I'm not advocating AI bans, that would only mean that it's the governments' or criminals' AI that would kill us, maybe a few years later. Or earlier, given what it might be instructed to do by such organizations. But anyway. As I said, I don't have, and can't have, a solution. Maybe AI is the Great Filter of the Fermi paradox.

Sorry for being so doom-and-gloom, but in my opinion, it's a very real threat.
Avatar
Guy Swann 2y ago

That’s a valid point. I tend to be optimistic though. Humanity has created psychotic and seemingly insane technologies before and we’ve made it through. This is obviously a bit unique, but so was every other example in their time. I tend to think humanity is more resilient than we give credit. That doesn’t mean that there isn’t the real possibility of a serious catastrophe or “great filter” event.

Reply to this note

Please Login to reply.

Discussion

Avatar
ElectronicsQuestions 2y ago

Yeah, one of those technologies is fiat money... We have a solution now, that could solve that problem, but despite owning what I currently consider a moderate but decent amount of said solution since around the first halving (more back then, of course, I was mining and buying stuff with it), and having listened to probably 100-200 hours of podcasts, read one-and-a-half of Saif's books, etc. etc., I'm still not anywhere near convinced that hyperbitcoinization will inevitably come, or that the value won't go to near 0. (Don't worry, I'm hodling an amount I've determined, just in case.) I guess I'm a pessimist - I'll attribute that to, particularly me being very privacy focused, learning in fiat world that change generally is in the negative direction...