It should be based on a trusted npubs list, not follows.

It probably still ends up with the same issue, but maybe it makes it less likely? Make more friction to become a trusted npubs. A follow is a single click, and means nothing. Theoretically marking someone as trusted wouldn't be done for someone you just encountered. My trusted list would be reserved for people I know IRL and for people I've had longer interaction with.