In Ancapistan, how would there be law and order? Why wouldn't people just rob and steal?

Anarchist handbook, chapter 13(ish) - a vision of private law enforcement:

If someone steals from you, you're naturally entitled to use force to defend your property, and to reattain it later.

By extension, your agent can too.

There would likely be detective agencies for hire. They would go get your stuff back (or payment for your damages).

But violence is expensive, and it'd likely be cheaper to settle the matter in a (non-government) arbitration forum. Forums would be incentivized to be fair or else no one would do business with them (unlike the current quasi monopoly on dispute resolution). And it'd be in the theifs interest as it avoids force against him, and gives him a chance to defend himself.

Likewise, detective agencies are incentivized to minimize harm to perpetrators as they could be pursued for damages by the same mechanism (and they lack the immunity that the current enforcers have)

And victims have the same incentive to choose responsible detective agencies that won't expose them to liability for damages they cause (since the detectives are acting as the agent of the victim)