Just because it's legal doesn't mean it's right. If you knowingly take a picture of someone you know doesn't want you to and you don't give them a chance to avoid it, you're a fucking asshole.
If it happens by chance then that it on them. But to secretly do it with intent is absolutely an asshole move.