The sentiment behind this type of talk is nice, but it ignores a reality for many. That is, if you were raised around violent or toxic folks you probably give off unsettling vibes to "normal" people. So normal healthy people can't quite put their finger on it, but back away slowly from you. You know who doesn't? The people who also have that violent or toxic vibe normalized in their lives, they'll come towards you to be a validating victim or a violence perpetrator or often both. It becomes a vicious cycle which reinforces alienation from the people around us.
I spent many years being a really strange and toxic person. When I started to get better I spent many years almost completely alone, and in many ways I'm still there. I am still deeply strange, and I have a sincerely hard time understanding what others think about anything let alone me, but to say I don't care is inaccurate. Not caring what others think is a bizarre goal.
Humans are social animals and it matters what your boss thinks about you, what your neighbors think about you, or what that attractive person yonder thinks about you. There are real life ramifications for these things. To pretend that's not the case is not a path towards navigating reality.
But sure, if you've been a lying manipulative people pleaser with no sense of your own personal identity then sure it's maybe a good step to take to try not to care what people think about you 🤣