That's quite a zero-sum view of wealth. Me gaining freedom only means a loss for whoever has control over me, *and* whether or not that "perceived" control is still valuable to them (they can only really control me by directly pointing a gun to my head).
There was a point in time when control of others was the only truly valuable thing; the only thing that made you wealthy. Since Smith, we've seen a rapid departure from that power structure due to specialization and trade. Permissionless and borderless payments will only add fuel to that fire.