the auth request (challenge) is sent from a relay to the client, this value is meaningless to anyone else
only the client can sign it, and anything else in the message is irrelevant
nobody can sign with the valid public key and challenge except the holder of the private key, so why does the relay address matter?
i already got one ack on this proposal and i am pretty confident the rest will ack also
and it solves the problem that makes your project more or less useless if you care about not exposing your nsec to remote attackers to hardware not in your direct physical control
this will mean that you can add one small feature to your project where it forwards AUTH messages back and forth to the relays, it only needs to keep track of the challenge and relay address so it forwards the auth response to the relay that will recognise it (and btw, typically these live for only 20 seconds at best).