There was a paper by Microsoft on how "safety training" LLMs measurably reduces their performances on all tasks, this is that.