I’ve come to the provisional conclusion that EA scaremongering about AI is a case of full projection. In other words, EA morality IS unaligned AI.

EA is basically warmed-over utilitarianism, in which the pursuit of a single “world-historical” objective makes all other considerations—moral, logistical, relational—unnecessary.

This is no different from the classic AI “paperclip problem”, in which an AI is given instructions to manufacture as many paperclips as possible and cannibalizes all of Earth’s resources to do so.

How is this meaningfully different from SBF deciding that humanity’s survival depends on electing Democratic politicians in the U.S. in the very near-term, and that the shortest path to that outcome is him personally amassing the largest possible fortune in the smallest possible amount of time, even at the cost of defrauding millions of people, betraying his team and closest friends, and catalyzing a further collapse of trust in America’s political institutions?

SBF was the AI in the paperclip problem. A single objective overrode every other; he had no capacity for self-reflection; and the world, to him, was completely flat: every moral good could be quantified and compared using the same rubric.

While that made him completely “unaligned” to the greater good of humanity, he *was* fully aligned to the objectives and worldview of his creators—his parents—who embody the particular elite amorality of a hyper-educated and politically-connected overclass that presumes their continued pre-eminence is humanity’s most pressing concern.

It is these people, EAs, who now say that only they can save the world from “unaligned AI”, that is . . . themselves.

We must not believe them. Morality, like human flourishing, emerges bottom-up, not top-down. The best and most beneficial AIs will be trained this way as well. The relationship between humans and machines does not have to be one of dictators ordering machines to amplify their own power. We can choose to nurture machines the way caring and skillful parents nurture children—as thoughtful, independent beings in relationships of love that provide the “ground” for all moral reasoning.