Money influences culture a lot. If suits were affordable, more people would wear them. If you could afford to wear a suit to go shopping then you must have at least 4-5 suits. People nowadays are lucky to own one suit. A lot of them just rent for special occasions.
Discussion
You can buy a suit off the rack for a couple hundred bucks. It's a cheap suit, but you can. Carhartt hoodies are basically the same price as a cheap suit coat and shirt. I just think it's more likely that people don't wear suits because they don't want to rather than they can't afford to.
I own 4 suits, I don't wear them because I don't like them, not because I can't afford to