Yeah this is pretty bad UX. Wallet designers should consider this because it will scare the crap out of noobs who don’t know what’s going on.
Discussion
This also seems pretty easy to mitigate too. Load the other common derivation paths in the background and check for balances. Alert the user if there is another balance on a different path. If not, do nothing and just clear the other public keys from memory.