The people who suffer the least amount of diversity will develop the least amount of empathy. They are also the ones most likely to find their way into positions of power.
Discussion
You can also go through lots of adversity and not develop empathy. I think it depends on how we view the adversity that determines if we feel the need to empathize.
"To those human beings who are of any concern to me I wish suffering, desolation, sickness, ill-treatment, indignitiesâI wish that they should not remain unfamiliar with profound self-contempt, the torture of self-mistrust, the wretchedness of the vanquished: I have no pity for them, because I wish them the only thing that can prove today whether one is worth anything or notâthat one endures." - Friedrich Nietzsche, The Will to Power