I don't know for sure, but the assumption is that the person being stared at feels like a part of them is being identified as odd/abnormal/strange ..

"Stop looking at my *insert body part one has a complex about*"

Regarding the person doing the staring, I think it's a primal thing. Almost like one is still on the lookout for anything that may harm our tribe.