Are the credit card charges really as big of an issue as people claim?

I ask because when I freelanced, it was just the cost of doing business. I wrote it all off as expense and in some cases just baked it into the prices. I never really lost as a service provider. Couldn’t everyone just bake the fees into the cost of the thing they are selling?