Sometimes they were taken for religious reasons, or because they were born into the wrong race. Even longer ago they were stolen from one trible by another because their parents were sturdy and looked like they might produce good fighters.

Sadly it's in our very nature to take what we want and justify it later. Maybe we'll get better, one day...