What other people do impacts their wellbeing. Because I am a human being, I'm a social creature and therefore my wellbeing is dependent on the wellbeing of others.
Even if I was a selfish asshole who didn't care about anything but myself, I would still have a reason to care what others do in the privacy of their bedroom.
As it happens, I am not a selfish asshole ans I genuinely want the best for others, so I'm against harmful sexualities not just for my own sake but for the sake of others.