In the context of using bip85 (as above) I'd say it's a pretty good idea. If it's a single signer then it depends how confident you are that the seed phrase is in a secure location where it won't be compromised. Finally, if it's part of a multisig then I would say it's too much complication at that point.

When in doubt, keep it simple to prevent locking yourself out. When using a passphrase, keep a physical copy of it separate from the seed phrase & device so you can refer to it if you need to.