Just needed to make sure I was using the private key associated with the xonly key