Here's perhaps the most fundamental point about why I don't tie my political/ideological beliefs close to my sense of who I am: because I'm not personally convinced that I'm not wrong about anything.

I'm a Bayesian. And my political views have shifted in sufficiently dramatic ways quite a few times in my life, that I'm pretty allergic to the idea that I know for certain that my beliefs are correct or optimal.

I am fundamentally open to the idea that I'm completely full of shit. And it's one of the reasons I'm so willing to debate and accept challenge. Out of a principle, I feel committed to the necessity of changing my mind if a better argument comes along.

So restricting myself to people who politically and ideologically agree with me, would cut me off from the possibility of gaining an insight that reveals the wrongness of my beliefs. This, to me anyways, seems stupid on its face.