If anyone knows the private key to your Bitcoin, then as far as Bitcoin is concerned, they are the rightful owner of those coins.