Adding a passphrase to a seed generates an entirely new wallet, so either way you would need to send your funds to the new wallet.

You are better off leaving the existing seed on Ledger (since it may be compromised), generating a new seed on a new wallet, and then transferring funds to the new wallet.