Sure, it imposes a physical cost on opponents. I think the point was made during an interview on the WiM Show that in the real world the line between offense and defense is extremely blurred. Walls and fortresses can be used offensively and armies can be used defensively.