If you hide your money from the government and then spend it, if they catch you they throw you in prison. I would prefer to remain out of such situations.
Would I prefer to keep my wealth secret from the government and pay no taxes? Sure. But I can't do much for my family if incarcerated.
It's one thing if you've got no money and you save what scraps you have privately. But I am not in that situation. I have enough wealth to be buying properties and businesses. That game ends if I commit tax fraud and end up in prison.
Tax fraud is no joke, and anyone with substantial wealth that I've never met pays their taxes.