You can also go through lots of adversity and not develop empathy. I think it depends on how we view the adversity that determines if we feel the need to empathize.

"To those human beings who are of any concern to me I wish suffering, desolation, sickness, ill-treatment, indignities—I wish that they should not remain unfamiliar with profound self-contempt, the torture of self-mistrust, the wretchedness of the vanquished: I have no pity for them, because I wish them the only thing that can prove today whether one is worth anything or not—that one endures." - Friedrich Nietzsche, The Will to Power