This is where the 100% Libertarians loose me. Securing a nation's border is one of the only few probably legitimate reasons for "the state" to exist. I agree that most immigration policies are crap, but, I wouldn't let vagrants stay on my property, especially if they are violent thugs. Secure borders are necessary for a high trust, low law count society that I'd prefer to live in.
Discussion
I've come to a similar conclusion... private property is the basis of a civil society; organizations such as the state are formed to ideally protect that property.
Edgelord libertarians will deny every instance that the state could provide value to it's citizens. The real world tends to be shades of grey and dirty compromises; but fundamentalist of any ideology believe a utopian ideal can be achieved no matter how ludicrously terrible that system would be in practice.