There are many possible designs, but I all of them, they would follow you ID (imagine a nip05 with multi key support), not the keys. Nip05 could change the key set based on the caller and could even encrypt to the followers PM. The important part is that the protocol is the same.People can build any ID security they wish to use in nostr.