If you whitelist a key before your attacker then no, you can move to a new key and your attacker can’t.

Hence the question 😅