Kind of stupid to say that because you obviously don't trust the government either if you're using Bitcoin

You're right they can't, but they can confiscate your property, throw you in a cage, or worse in some places, if you don't give them your keys. 99% of people would give in especially if they have a family.