People often only think of freedom from physical violence. While that is the worse there are certainly other freedoms that can be restricted.

Physical restriction leads to an opposite response of force and doesn't last as long as subjugation or enslavement.

This is why Jefferson considered the banks more dangerous than the army, the army has the power to kill, but the banks have the power to enslave.