Requiring Authentication (NIP-42) to get DMs (kind4) would improve a bit the privacy.

It's no guarantee that nobody could access your metadata, but it's an improvement over what it's now.