Suggestion: after first message with someone, tag posts not with their public key, but a key derived from shared secret.

That hides further messages from the public eye. Basically, only the initial message request is public.

tag: hash(shared secret + their public key)