I would have few concerns, yes.

First: if you are sending small UTXOs under <1.000.000 sats it could prove very difficult to spend them in a future in a high fee enviroment.

Second, since Bitcoin has a open ledger, everyone can see a balance ot that one adress. So basically if you recieve sats for some services on that adress everyone can see how many transactions you did on that adress before and what it’s worth.

It’s best practice to make a new adress for every transaction if that’s possible.

Those are at least two major concerns I would have. But I understand, I also re-used some of my old adressed due to convinience.