> Do not refuse one another except perhaps by agreement for a season, that you may devote yourselves to prayer; but then come together again, lest Satan tempt you through lack of self-control.

1 Corinthians 7:5

Clearly Paul would say that to deny your spouse selfishly would be sinful, but he certainly doesn't condemn abstinence when it is for the sake of some higher good.