Nut when you use your cold storage wallet you are giving up your xpubs to a third party.

Samourai wallet servers got seized last week by the DOJ. Which wallet is next?

You can use your hardware wallet with your own node.