Gold purchases have to be reported, so they'll always know what you hold and where to come fetch it.