You can't do it at the user level, but yeah private, no broadcast relays would be the way to do it. Lots of reason to do so, but i guess not a developer priority at the moment.
Discussion
I feel like nostr:npub10npj3gydmv40m70ehemmal6vsdyfl7tewgvz043g54p0x23y0s8qzztl5h could figure it out 🤞
