It depends. If you want the SSH and GPG keys on the backup Yubikey you have to generate your master encrypt/sign key on an air-gapped machine and then create a backup of the .gnupg folder.

Generate your sub-keys for the first Yubikey.

Transfer the master key and sub-keys to the Yubikey.

Restore the .gnupg folder from the backup.

Generate your yubikeys for the second Yubikey and transfer the master key and sub-keys to the second Yubikey.

Every time you transfer keys to a Yubikey, you end up with stubs for those keys in your air-gapped machine, that’s why you need to make a backup of the .gnupg folder.

You could do this before or after it doesn’t matter, but you should also export your master key and encrypt it with a passphrase and keep it secure. You can buy one of those rugged USB memory sticks or YOLO it with a micro sdcard. You should do this even if you don’t have a second Yubikey.

Make sure you wipe that air-gapped machine once you are done!

If you only want to use the second Yubikey for signing into your password manager or something you can skip all that above and just register the second key.

Some security experts will frown at my recommendation because I didn’t suggest something so just DYOR and find what’s the acceptable level of security that YOU want.