Yes the reason damus only connects to specific relays is because the gossip model allows other users to make your client fetch from any relay, potentially rogue ones that could send bad data and data that may not match your queries.

Damus android/notedeck only ever queries the local relay (nostrdb) so queries are always well behaved. Damus iOS is switching to this model soon as well.