They use phone numbers because if you have someones phone number you usually have met/had contact with that person in meatspace and can evaluate if this is a trustworthy person you would recommend meeting for a BTC-Fiat-exchange. How would you get this kind of trust level for anon npubs?

Sure this is a tradeoff. Can you think of a better way to keep the trust level high?