Large language models can do jaw-dropping things. But nobody knows exactly why.
Two years ago, Yuri Burda and Harri Edwards, researchers at the San Francisco–based firm OpenAI, were trying to find out what it would take to get a large language model to do basic arithmetic. They wanted to know how many examples of adding up two numbers the model needed to see before it was able…