After skimming through the NIPs, I didn't actually see any explicit wording for how relays talk to each other.

I imagine (again, having not done any actual coding on Nostr yet) that a relay could just act as a client to another relay and send/receive a bunch of messages that have already been signed by completed unrelated entities.

A message is just a message that I can pass around, whether I'm the one who created and signed it, or if it was created and signed by someone else. As long as the message ID checks out, then it's a valid message.

See https://github.com/nostr-protocol/nips/blob/master/01.md#events-and-signatures