Right now relays are one of around five or so implementations. The issue is at some point when we end up with many more satellite (smaller) relays, malicious relays (or even software bugs) will start to appear.
Maybe trusted relays could have their events skip validation - however, at any moment validation could become really important to enforce client side.