Emergent properties of Large Language Models (LLM)
The Unpredictable Abilities Emerging From Large AI Models
Large language models like ChatGPT are big enough that they're displing startling, unpredictable behaviors
https://www.quantamagazine.org/the-unpredictable-abilities-emerging-from-large-ai-models-20230316
Discussion: https://news.ycombinator.com/item?id=35195106
Researchers at Google Brain showed how a model prompted to explain itself (a capacity called chain-of-thought reasoning) could correctly solve a math word problem, while the same model without that prompt could not.
...
... (cont'd)
[Google Brain] Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
https://arxiv.org/abs/2201.11903
Recent work suggests chain-of-thought prompting changes scaling curves & therefore the point where emergence occurs.
In their paper the Google researchers showed chain-of-thought prompts could elicit emergent behaviors.
Such prompts, which ask the model to explain its reasoning, may help researchers begin to investigate why emergence occurs at all.
...
... (cont'd)
Recent findings like these suggest at least two possibilities for why emergence occurs.
1. In comparison to biological systems larger models truly gain new abilities spontaneously.
2. What appears to be emergent may instead be the culmination of an internal, statistics-driven process that works through chain-of-thought-type reasoning. Large LLMs may simply be learning heuristics that are out of reach for those with fewer parameters or lower-quality data.
...
