Hmm. I always assumed you could either have a single xpub per seed phrase, or have multiple “wallets” by adding a passphrase to a seed - which I believe generates a new xpub. But this may be a bit beyond my technical know-how, at this point.

To your original question, the reasons not to give your xpub to swan include the privacy concerns I mentioned. If you move those utxos out of cold storage later (or just move to a different wallet), you can gain forward-looking privacy via coinjoin but Swan/onlookers would know you moved those coins.