no, the Fediverse does not cache all images by default, You have to configure it to do that. Many do not because of what you said and bandwith
Discussion
fediverse is inbox model, nostr is outbox. both have their strengths and weaknesses. I like smart clients and dumb servers, fediverse is dumb clients and smart servers. I think smart clients give the most power to the users, which is why its superior for freedom and digitial sovereignty without needing to be technical.