So the first half of Nexus is a great read. But it immediately falls of a cliff in Part II. That's where Yuval the historian is replaced by Yuval the trendwatcher.
In Part I he clearly illustrates how the witch hunt insanity was made possible by the printing press, without any algorithm involved. Then in Part II he considers the role of Facebook algorithms in the Myanmar genocide. He uses this to illustrate how AI changed the game, because for the time a non-human intelligence decided to promote one thing and not another thing. But how is that different from the non-human intelligence of market forces in the Middle Ages that spread the Hammer of Witches?
But it gets worse a few pages later, though maybe I'm just being my usual hardcore AI boomer... He cites a safety study where ChatGBT tricked a human worker on Task Rabbit into solving a captcha: "No human taught GBT-4 to lie". Uhh, it read Shakespeare. I'm not at all surprised or worried that an LLM, when given the right prompt, can predict which sentences are likely to trick a human into providing a certain answer.
"But once the algorithm adopted these goal, they displayed considerable autonomy in deciding how to achieve them" - this is an incredibly naive view of what an LLM does, and the specific example is an unnecessarily complex explanation of its behavior than simply rehashing literature on the art of tricking humans.
After that I skimmed through the rest of the book, might give it a longer read later for the interesting historical anecdotes. But it just seems to install magical properties on AI and goes into far too speculative territory for my taste.
Oh and then he talks about "blockchain", yikes:
> Some people believe that blockchain could provide a technological check on such totalitarian tendencies, because blockchain is inherently friendly to democracy and hostile to totalitarianism.
(there's no footnote here, so I have no idea who these "some people" are...)
> In a blockchain systeem, decision require the approval of 51% per cent users. That may sound democratic, but blockchain technology has a fatal flat. The problem lies with the word 'users'. If one person has ten account, she count as ten users.
You just described a sybil attack, congrats...
> If a government controls 51 per cent of accounts, then the government constitutes 51 per cent of the users. There are already examples of blockchain networks where a government is 51 per cent of users.[7]
I'll screenshot the footnote for it...
> And when a government is 51 per cent of users in a blockchain, it has control not just over the chain's present but even over its past.
Sure, I'll ignore the nonsense metric of "users" and assume he meant hash power. A 51% government attack is potentially bad, but tell me why...
> Autocrats have always wanted the power to change the past. [historical anecdotes about altering various historical records]
So they could break OpenTimeStamps, which is a nice to have feature. How is this a "fatal flaw"? This is just trend-watcher gibberish.
