"But without government, society would not be able to organize as well as it does, and people would prosper less."

Even if that were true (I'm not convinced it is) the ends do not justify the means.