Can you even have intelligence in the absence of an embodied consciousness connected to the world via the senses? I don't know the answer to that, but I'm pretty sure the kind of nerds building this phoney AI shit wouldn't even understand the question.
Discussion
If you believe this sort of thing is possible, you probably believe Helen Keller was legit.
#[0]
I haven't read his book, but I listened to the interview linked below and he's exactly the kind of AI researcher I'm talking about. He has some interesting things to say about the social implications of AI but when asked about the nature of consciousness he resorts to pure drivel.
God (if he exists) is at least that. There's at least an argument that intelligence doesn't have to be embodied given that Math, logic, etc. are metaphysical.
True, but computers have been handling math and basic logic for a long time now. The promise of AI is something different. It's advocates are slippery when it comes to definitions, but they are clearly talking about self-awareness and volition. I don't see how this is possible.