Yeah you might want to consider to have them either in separate accounts or separate wallets, separate passphrase is another option.

If you have a ColdCard the bip-85 spec might be another good method.