If you enter a friends house, store or work. No one cares. If you enter a respected place, like your parents house, a place of worship, higher business, government... etc. You remove your cover. Women are exempt from having to remove their headware.

It's considered rude, and disrespectful, to not present your face when entering someone's home, in the West. Other cultures may have other rules, or no rules, et al.