As long as the Iris version doesn't allow anyone with your keys to change it, it should be fine. The thing you want to avoid is to tell all your friends the iris address IS you and then your keys leak and the attacker not only gets your keys but also your NIP-05. All your friends will think they are talking to you. Not only because it comes from your PubKey, but also from a valid NIP-05.

Ideally the password that allows changing the NIP05 is completely unrelated to your keys, managed completely separately.