Ok so the best way to solve this is using a dedicated account of your wallet and deducting from child extended public keys. In this way only a subset of your public keys will be exposed.