The fear that it might cost too much to send an on chain transaction in the future than what a UTXO is worth. Like would you send $10 if it cost $100 to do so? This is a hypothetical situation though. No one knows how the fees will look like that far into the future.
Discussion
This is not exactly solved if you have a larger UTXO since you will still pay the same amount in fee. Higher fees make sending lower balances not worthwhile regardless of the size of the UTXO you use as an input. That said having less inputs will lower the total fee. The way I see it the main concern when having smaller utxos is losing privacy if you need to combine them. Also for accounting purposes it’s better to have less UTXOs, imagine having 40 $1 bills in your wallet.