Not the DM content, but all other metadata (who you are messaging and when) is effectively public. If you take an npub and log into a client like snort.social you can actually see the inbox (without message content) of any account