The disconnect here is a misunderstanding of the difference between positive and negative rights.

There's a reason why the Libertarian party's mascot is a porcupine.

The porcupine is not a predator. It's not an aggressive animal. But if you invade it's personal space, it will fuck you up son.