The first grand surprise of LLMs

The first grand surprise of LLMs is that the intelligence we experience in AI derives from the intelligence we have inadvertently coded into human text, rather than from any explicit software code. There appears to be a seminal, fundamental relationship between language and thinking. Human writing is thus not only a reflection of the structure of language, but to some degree also a reflection of human thinking. Distill the patterns in human writing at scale, and you also get some patterns of human thinking. Imitate human writing and conversation, and you can imitate human intelligence — at least in part. -Kevin Kelly