Yes, hurt performance all around. Relays may filter low WoT content, like a spammy npub commenting on tons of posts of those who don't "trust" it. I think it'd need a threshold, though, because it would otherwise lock new users out.
Discussion
WoT seems to have a lot of potential, but it's not a silver bullet. We need algorithms or models to promote discoverability for new npubs.
nostr:note1fedfwhv9d7ths9ky2l8k3r93u23e69fsds0r8tzrfnw64l7unx3qy0yunt