They could, but then You’d have to share your xpub to allow others to associate all your child keys with your unified ID. Not optimal