Not necessarily. For thousands of years self sovereign villages and towns had signs that denied entrance by certain individuals based on distinguishing factors. I suggest reading Hans Hermann Hoppe and Rothbard for more background on this topic.