The most common explanation for this is actually that you were mentioned in a DM.
Bit sloppy, but some clients pull these since your pubkey appears in them but you can’t decrypt the message as you are not the recipient.
Discussion
I think clients should not include those tags when writing DM events but at least some of them do.
Agreed, can't think of a reason for pubkeys to need to be registered in private messages
That does makes sense. Guess there's no talking behind people's back on nostr lol!