Yes, selection pressure comes from the female, but women tend to share males. That's why humans have significantly fewer male than female ancestors.

Males with high status therefore often have/had offspring with multiple women. Some even with hundreds or thousands of women. Those men were each their own evolutionary bottleneck, so we're a reflection of their choices.