Your reputation is your true private key. Your reputation is what gives your #Nostr keys any value to others in the first place, not the other way around. Nobody follows you because they like your npub; they like you, and so they follow the npub you use. Protect your keys, of course, but it is very easy to generate a new key pair if your old one gets compromised. Be more concerned about your reputation and the value and quality of content you are providing to others. If someone switches to a new key, it will get shared around, and if you wanted to follow them before, you will want to follow them again. Rebuilding a lost follower list is a lot easier than rebuilding a lost reputation.