Inflows vs outflows. If my investments succeed then i would be limited in my ability to spend them. I would arguably be less sovereign because i would have to operate via some proxy between the gray market and white market. Paying rent, or a house payment would be difficult. Sure I could probably live as a minimalist, but i would still depend on other people to function. My lanlord has to run a credit check, proof of employment, etc. Live like a drug dealer?
Discussion
The value prop of Bitcoin/Monero is permissionless which is incompatible with white markets by definition...so yes, pretty much. Unless you can find clever ways to navigate.
If one is going to make transactions that follow the rules, there is little reason to have or use Bitcoin/Monero. In fact, it might even end up worse than just using fiat i.e. what you went/are going thru
I think the issue is my environment. I have been surrounded by sheeple my whole life, and have never been able to successfully operate in the gray market without it blowing up in my face. The whole prisoner's delimma dynamic only works in your favor when the mafia has your back.

Shit just blows up in your face. Nothing is perfect, like you said. You play by the rules, you get burned. You work around the rules, get burned. There are no right answers.