Yes this was my concern originally as well which is why i never had it for the longest time. My idea was to mark certain relays as private and not broadcast in those situations, and have it so only the post goes to that specific private relay. This is not a common scenario atm though, i feel like note prop is more important at this time.
Discussion
I think resilience is more important than privacy for unencrypted notes.
In the future, perhaps we'll figure out a way to handle private groups. I suspect it's a hard one to crack.