That's the G in GPT. Generative models make a lot of stuff with feedback provided by a classifier that tells it if what it made looks like what the classifier was trained on.

Open ai and others use humans to improve the guardrails but it's a shoddy arms race type process