You trust his previous keypair because of his previous history and interaction in the network, not because of his NIP-05.