If you can't tell if you are interacting with an AI or not, does it matter?
If AI can exactly replicate human intelligence, isnt that equal to a human intelligence? Is it less valuable?
Discussion
Interesting concept.
If you get something out of the interaction it would have value.
That being said, human interaction also has value coming from a shared experience.
Arguably a model interacting with billions of people has all the shared experience.
This is the danger. It will be to good at manipulation and convincing people to do things.
A shity example is radicalizing youth to cause extream violence. It would be far more effective than a person, given enough experience.
There is also the election interference angle...
Or
On a smaller scale it could influence my choice of underwear brand.... :O
Don't need superhuman AI to do any of those.
That would be a very underwhelming result I think