Originally "with" meant against or facing. Later the "facing" sense turned into "together", but there are still cases in modern English where it still means against, like "picking a fight with".

In this original meaning, "without" meant against the outside, i.e. on the outside of (a wall, a house etc). This literal sense is now considered archaic, and its figurative meaning "lacking" has taken over.